Computer hardware system and artificial intelligence computing process

Abstract

To provide a computer hardware system and a process for effectively improving performance and stability and extending a service life by using resource sharing.SOLUTION: In the artificial intelligence operating system 100, the computer hardware system 120 executing the artificial intelligence computing process 110 includes a primary storage device 125, a primary processor 124, a secondary storage device 127, and at least one accelerator processor 128. The primary storage device stores write-intensive data, the primary processor is connected to the primary storage device and executes a setup process in an artificial intelligence computing process, the secondary storage device stores read-intensive and non-read-intensive data, and the accelerator processor reads the read-intensive and non-read-intensive data stored in the secondary storage device and accesses the write-intensive data in the primary storage device via the primary processor.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Cross-references to related applications and priority claims This application claims the benefit of Taiwan Patent Application No. 113126792, filed with the Taiwan Intellectual Property Office on July 17, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The present invention relates to artificial intelligence technology, and more particularly to a computer hardware system and an artificial intelligence computing process executed by the computer hardware system that utilizes resource sharing to effectively improve the performance and stability of the computer hardware system and extend the service life of the computer hardware system. [Background technology]

[0003] With the rapid development of artificial intelligence (AI) technology, its range of applications is continuously expanding from image recognition and speech recognition to fields such as natural language processing, and advances in AI technology are transforming our lives. However, these technologies rely on massive computational resources and storage capacity. The learning and prediction processes of AI computing require large amounts of memory to store AI model parameter data and cache data generated during calculations, which not only increases equipment costs but also impacts the management and optimization of computational resources. In this situation, resource sharing becomes an important means of solving problems.

[0004] First, we need to understand the memory demands of AI computing. Modern AI models, such as deep neural networks, typically contain millions or even billions of parameters. These parameters must be stored in memory for fast access and updating during training and prediction. Furthermore, a large amount of intermediate data (cache data) is generated during the computation process, which also needs to be stored in memory during the computation process. For example, training a large-scale natural language processing model (such as GPT-3) can require hundreds of gigabytes or even terabytes of memory space.

[0005] In addition to managing and allocating computing resources, resource sharing also requires consideration of the characteristics of various storage devices. Solid-state drives (SSDs) such as NAND flash memory have limited read / write cycles, and prolonged and frequent read / writes will deplete memory resources and shorten the device's lifespan. Therefore, the resource sharing process must take these device characteristics into account to ensure system stability and long-term operation.

[0006] Therefore, there is an urgent need to provide a computer hardware system and an artificial intelligence computing process executed by the computer hardware system, as well as a need to use resource sharing to effectively improve the performance and stability of the computer hardware system at a lower cost and extend the service life of the computer hardware system. Summary of the Invention

[0007] The objective of the present invention is to provide a computer hardware system for executing artificial intelligence computing processes that solves the problem of memory space shortage at a relatively low cost through resource sharing, and improves the performance and stability of the computer hardware system, which is particularly important for AI computing that requires intensive processing of large amounts of data.

[0008] To achieve the above object, in one aspect, the present invention provides a computer hardware system for executing an artificial intelligence computing process, the computer hardware system comprising: a primary storage device, a primary processor, a secondary storage device, and at least one accelerator processor. The primary storage device is configured to store instructions for executing the artificial intelligence computing process and write-intensive cache data. The primary processor is connected to the primary storage device and configured to execute a setup process in the artificial intelligence computing process and access the write-intensive cache data in the primary storage device. The secondary storage device is configured to store read-intensive and non-read-intensive data including artificial intelligence model data. The at least one accelerator processor is connected to the secondary storage device and the primary processor. The at least one accelerator processor is configured to load the artificial intelligence model data stored in the secondary storage device, perform multiple layer calculations in the artificial intelligence computing process based on the artificial intelligence model data, and access the cache data in the primary storage device via the primary processor. The setup process performed by the primary processor includes the steps of defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor, inputting data and preprocessing the data, and importing the preprocessed data into the plurality of layer computations.

[0009] In another aspect, the present invention further provides an artificial intelligence computing process executed by a computer hardware system, the computer hardware system including a primary storage device, a primary processor, a secondary storage device, and at least one accelerator processor. The artificial intelligence computing process includes the steps of: accessing write-intensive instructions in the primary storage device using the primary processor to perform a setup process; loading read-intensive and non-read-intensive data including artificial intelligence model data from the secondary storage device to the at least one accelerator processor; and executing multiple tier calculations in the artificial intelligence computing process using the at least one accelerator processor based on the artificial intelligence model data, accessing cache data in the primary storage device via the primary processor. The setup process includes the steps of defining the multiple tier calculations and assigning the multiple tier calculations to the at least one accelerator processor; inputting data and preprocessing the data; and importing the preprocessed data into the multiple tier calculations.

[0010] In another aspect, the present invention further provides a computer hardware system for executing an artificial intelligence computing process, the computer hardware system including a primary storage device, a primary processor, a secondary storage device, and at least one accelerator processor. The primary storage device is configured to store write-intensive data. The primary processor is connected to the primary storage device and configured to execute a setup process for the artificial intelligence computing process. The secondary storage device is configured to store read-intensive and non-read-intensive data. The at least one accelerator processor is configured to read the read-intensive and non-read-intensive data stored in the secondary storage device, execute multiple tier calculations for the artificial intelligence computing process based on the read-intensive and non-read-intensive data, and access the write-intensive data in the primary storage device via the primary processor. The setup process executed by the primary processor includes the steps of defining the multiple tier calculations and assigning the multiple tier calculations to the at least one accelerator processor, inputting data and preprocessing the data, and importing the preprocessed data into the multiple tier calculations.

[0011] In summary, the computer hardware system of the present invention and the artificial intelligence computing process executed by the computer hardware system effectively improve the performance and stability of the computer hardware system through resource sharing, and extend the service life of the computer hardware system. [Brief explanation of the drawings]

[0012] The above objects and advantages of the present invention will become more apparent to those skilled in the art by reference to the following detailed description and accompanying drawings. [Figure 1]1 is a schematic diagram of a computer hardware system that executes an artificial intelligence computing process under an artificial intelligence operating system in a specific embodiment of the present invention. [Figure 2] 3 is a flowchart of an artificial intelligence computing process in a specific embodiment of the present invention. [Figure 3] 1 is a flowchart of a setup process for an artificial intelligence computing process in a specific embodiment of the present invention. [Figure 4] 10 is a flowchart of a setup process for an artificial intelligence computing process in another specific embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following detailed description should be read with reference to the accompanying drawings of the present disclosure. The accompanying drawings of the present disclosure are provided as examples to introduce various different embodiments of the present disclosure and to help understand how the present disclosure can be implemented. The embodiments of the present disclosure provide sufficient content for those skilled in the art to implement the embodiments disclosed by the present disclosure or to implement embodiments derived from the contents disclosed by the present disclosure. It should be noted that the embodiments are not mutually exclusive, and some embodiments can be appropriately combined with one or more other embodiments to form new embodiments. That is, the implementation of the present disclosure is not limited to the embodiments disclosed below. Furthermore, for the sake of conciseness and clarity of description, relevant details are not excessively disclosed in each embodiment. Even specific details disclosed are used only as examples to facilitate understanding, and relevant specific details in each embodiment do not limit the disclosure of this application.

[0014] 1 is a schematic diagram of a computer hardware system for executing an artificial intelligence computing process under an artificial intelligence operating system according to a specific embodiment of the present disclosure. In FIG. 1, a computer hardware system 120 for executing an artificial intelligence computing process 110 in an environment of an artificial intelligence operating system 100 is provided, and the computer hardware system 120 includes a primary processor 124, a primary storage device 125, at least one accelerator processor 128, and a secondary storage device 127. The primary storage device 125 is configured to store instructions and cache data for executing the artificial intelligence computing process 110. The primary processor 124 is connected to the primary storage device 125 and is configured to execute a setup process for the artificial intelligence computing process 110.

[0015] In a specific embodiment, the artificial intelligence computing process 110 may be an artificial intelligence prediction application or an artificial intelligence training application. For example, when executing the artificial intelligence computing process 110 for an artificial intelligence prediction application, a setup process can be performed using the primary processor 124. Specifically, the primary processor 124 first defines multiple layer calculations of a model and assigns the multiple layer calculations to at least one accelerator processor 128. Next, data is input from the input terminal 121, and the primary processor 124 performs preprocessing such as standardization, normalization, feature extraction, and cleaning to ensure that the data conforms to the model's input format. The preprocessed data is then imported into the multiple layer calculations defined by the model. Once the setup process is complete, the artificial intelligence model data is loaded from the secondary storage device 127 to at least one accelerator processor 128. Because existing artificial intelligence model data typically contains millions or even billions of parameters, in this specific embodiment, these parameters are primarily stored in the secondary storage device 127. The secondary storage device 127 is implemented using a solid-state drive (SSD), a hard disk drive (HDD), NOR flash, RRAM, or FRAM (preferably NAND flash). However, solid-state drives have a limited number of read and write cycles. Long-term and frequent read and write cycles can damage solid-state drives and shorten their lifespan. Therefore, in the present invention, the secondary storage device 127 is primarily used to store read-intensive and non-read-intensive data, taking full advantage of the large capacity and low cost of SSDs and avoiding the drawbacks of limited write cycles. In a specific example, the read-intensive and non-read-intensive data includes artificial intelligence model data.

[0016] Next, at least one accelerator processor 128 executes multiple layer calculations in the AI ​​computing process 110 based on the AI ​​model data and accesses cache data in the primary storage device 125 via the primary processor 124. Finally, the computer hardware system 120 outputs the results of the AI ​​prediction application to the output terminal 122. In this specific embodiment, these cache data are primarily stored in the primary storage device 125, which is implemented with memories such as DRAM, SRAM, and MRAM. However, while such storage devices are relatively expensive, they can withstand frequent and long-lasting write operations. Therefore, in the present invention, the primary storage device 125 is primarily used to store write-intensive data, taking advantage of its durability and fast response time. In a specific example, the write-intensive data includes instructions and cache data for executing the AI ​​computing process. By allocating memory resources to different operations in the present invention, not only data processing efficiency but also the efficiency of the AI ​​computing process is improved.

[0017] As described above, in a specific embodiment, the artificial intelligence computing process 110 can be an artificial intelligence prediction application or an artificial intelligence training application. For example, when executing the artificial intelligence computing process 110 for an artificial intelligence training application, a setup process can be performed using the primary processor 124. Specifically, the primary processor 124 first defines multiple layer calculations of the model and assigns the multiple layer calculations to at least one accelerator processor 128. Next, before training the model, data is input from the secondary storage device 127, and the primary processor 124 performs data preprocessing such as standardization, normalization, feature extraction, and cleaning to ensure that the data conforms to the model's input format. Next, model training iteration parameters, i.e., the number of times (called epochs) the model is trained on the entire data set, are set. Typically, a certain number of iterations is set to ensure the model is fully trained. Once the setup process is complete, the artificial intelligence model data is loaded from the secondary storage device 127 to at least one accelerator processor 128. Because existing artificial intelligence model data typically contains millions or even billions of parameters, in this specific embodiment, these parameters are primarily stored in the secondary storage device 127. The secondary storage device 127 may use a solid-state drive (SSD), a hard disk drive (HDD), NOR flash, RRAM, or FRAM (preferably NAND flash). However, the read / write cycles of a solid-state drive are limited. Long-term and frequent read / write cycles can damage the solid-state drive and shorten its lifespan. Therefore, in the present invention, the secondary storage device 127 is primarily used to store write-intensive data, taking full advantage of the large capacity and low cost of SSDs and avoiding the drawbacks of limited write cycles. In a specific example, the read-intensive and non-read-intensive data includes artificial intelligence model data.

[0018] Next, at least one accelerator processor 128 executes multiple layer calculations in the AI ​​computing process 110 based on the AI ​​model data and accesses the cache data in the primary storage device 125 via the primary processor 124. Finally, the computer hardware system 120 outputs the results of the AI ​​training application to the secondary storage device 127. In this specific embodiment, these cache data are primarily stored in the primary storage device 125, which is implemented with memory such as DRAM, SRAM, and MRAM. While such storage devices are relatively expensive, they can withstand high-frequency and long-term write operations. Therefore, in the present invention, the primary storage device 125 is primarily used to store write-intensive data due to its durability and fast response. In a specific example, the write-intensive data includes instructions and cache data for executing the AI ​​computing process. By allocating memory resources to different operations in the present invention, not only data processing efficiency but also the efficiency of the AI ​​computing process is improved.

[0019] In certain embodiments, the computer hardware system 120 of the present invention further includes a storage controller 126 connected to a secondary storage device 127, the storage controller 126 being configured to determine whether the at least one accelerator processor 128 accesses the secondary storage device 127.

[0020] In a specific embodiment, the computer hardware system 120 of the present invention may further include an accelerator storage device 129 connected to and accessed by at least one accelerator processor 128. In a specific embodiment, the accelerator storage device 129 can be considered an extension of the primary storage device 125, i.e., it uses memory such as DRAM, SRAM, or MRAM to store primarily write-intensive data for access by the at least one accelerator processor 128. In a specific embodiment, the write-intensive data includes instructions and cache data for executing artificial intelligence computing processes. In a specific embodiment, the accelerator storage device 129 is also accessible by the primary processor 124 via the at least one accelerator processor 128, maximizing the benefits of resource sharing.

[0021] In a particular embodiment, the primary processor 124, storage controller 126, and at least one accelerator processor 128 of the computer hardware system 120 of the present invention communicate with each other via a PCIe interface 123. However, it should be understood by those skilled in the art that the present invention is not limited to the use of a PCIe interface 123.

[0022] In a specific embodiment, at least one accelerator processor 128 of the computer hardware system 120 of the present invention is a GPU, NPU, TPU, ASIC, etc., and is directly connected to a respective accelerator storage device 129 or can access write-intensive data in primary storage device 125 via primary processor 124 and read-intensive and non-read-intensive data in secondary storage device 127 via storage controller 126.

[0023] The present invention further provides an artificial intelligence computing process executed by the computer hardware system 120 in cooperation with the computer hardware system 120 of FIG. 1. The computer hardware system 120 includes a primary processor 124, a primary storage device 125, a secondary storage device 127, and at least one accelerator processor 128. FIG. 2 shows a flowchart of the artificial intelligence computing process in a specific embodiment of the present invention. In FIG. 2, the artificial intelligence computing process of the present invention includes the following steps: First, in step S21, the primary processor 124 accesses instructions in the primary storage device 125 to execute a setup process.

[0024] As described above, the artificial intelligence computing process 110 may be an artificial intelligence prediction application. FIG. 3 is a flowchart of a setup process for the artificial intelligence computing process 110 of the artificial intelligence prediction application executed by the computer hardware system 120 of the present invention. In FIG. 3, first, step S211 is executed, in which the primary processor 124 defines multiple layer calculations of a model and assigns the multiple layer calculations to at least one accelerator processor 128. Next, in step S212, data is input from the input terminal 121, and the primary processor 124 performs preprocessing such as standardization, normalization, feature extraction, and cleaning so that the data conforms to the input format of the model. Next, in step S213, the preprocessed data is imported into the multiple layer calculations defined in the model.

[0025] In another specific embodiment, the artificial intelligence computing process 110 may be an artificial intelligence training application. FIG. 4 is a flowchart of the setup process of the artificial intelligence computing process 110 for the artificial intelligence training application executed by the computer hardware system 120 of the present invention. In FIG. 4, step S216 is first executed, in which the primary processor 124 defines multiple layer calculations of the model and assigns the multiple layer calculations to at least one accelerator processor 128. Next, in step S217, before training the model, data is input from the secondary storage device 127, and the primary processor 124 performs data preprocessing such as standardization, normalization, feature extraction, and cleaning to ensure that the data conforms to the model's input format. Next, in step S218, the iteration parameters for model training, i.e., the number of times (called epochs) the model will be trained on the entire data set, are set. A certain number of iterations is usually set to ensure that the model has been fully learned.

[0026] Next, in step S22, the artificial intelligence model data is loaded from the secondary storage device 127 to at least one accelerator processor 128. Next, in step S23, the at least one accelerator processor 128 performs multiple layer calculations in the artificial intelligence computing process based on the artificial intelligence model data, and accesses cache data in the primary storage device 125 via the primary processor 124. In a specific embodiment, the at least one accelerator processor 128 can be directly connected to the accelerator storage device 129 for access by the at least one accelerator processor 128. In a specific embodiment, the accelerator storage device 129 can be considered an extension of the primary storage device 125, i.e., memories such as DRAM, SRAM, and MRAM are mainly used to store write-intensive data for access by the at least one accelerator processor 128. In a specific embodiment, the accelerator storage device 129 can also be accessed by the primary storage device 125. The primary processor 124 maximizes the effect of resource sharing through the at least one accelerator processor 128.

[0027] Next, in step S24, the primary processor 124 determines whether the calculations of all layers are complete. If so, the primary processor 124 obtains the calculation results of the artificial intelligence computing process (step S25) and outputs the calculation results from the output terminal 122 or stores the calculation results in the secondary storage device 127. If the primary processor 124 determines that the calculations of all layers are not complete, step S22 is executed again, and the artificial intelligence model data is loaded from the secondary storage device 127 into at least one accelerator processor 128.

[0028] From the above, it can be seen that the computer hardware system of the present invention and the artificial intelligence computing process using the same solve the problem of memory space shortage at a relatively low cost through resource sharing, and improve the performance and stability of the computer hardware system, which is particularly important for AI calculations that require intensive processing of large amounts of data.

[0029] The present invention has been disclosed by the above embodiments or examples, but the present invention is not limited thereto. Those skilled in the art can make appropriate changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention is defined by the appended claims. [Explanation of symbols]

[0030] 100: Artificial Intelligence Operating System 110: Artificial Intelligence Computing Process 120: Computer Hardware Systems 121: Input terminal 122: Output terminal 123: PCIe interface 124: Primary processor 125: Primary storage 126: Storage controller 127:Secondary storage device 128: Accelerator processor 129: Accelerator storage device

Claims

1. 1. A computer hardware system for executing artificial intelligence computing processes, comprising: a primary storage device configured to store instructions for executing the artificial intelligence computing process and write-intensive cache data; a primary processor connected to the primary storage device and configured to execute a setup process in the artificial intelligence computing process and access the write-intensive cache data in the primary storage device; a secondary storage device configured to store read-intensive and non-read-intensive data, including artificial intelligence model data; and at least one accelerator processor connected to the secondary storage device and the primary processor, the at least one accelerator processor configured to load the artificial intelligence model data stored in the secondary storage device, perform multiple layer calculations in the artificial intelligence computing process based on the artificial intelligence model data, and access the cache data in the primary storage device via the primary processor; The setup process executed by the primary processor includes: defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor; inputting data and pre-processing said data; and importing the preprocessed data into the plurality of layer calculations.

2. 10. The computer hardware system of claim 1, further comprising a storage controller coupled to the secondary storage device and configured to determine whether the at least one accelerator processor accesses the secondary storage device.

3. 10. The computer hardware system of claim 1, further comprising an accelerator storage device coupled to the at least one accelerator processor and configured to be accessed by the at least one accelerator processor.

4. 3. The computer hardware system of claim 2, wherein the primary processor, the at least one accelerator processor, and the storage controller communicate with each other via a PCIe interface.

5. The setup process executed by the primary processor includes: defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor; loading the preprocessed data from the secondary storage device; and setting an iteration parameter.

6. An artificial intelligence computing process executed by a computer hardware system, comprising: The computer hardware system includes a primary storage device, a primary processor, a secondary storage device, and at least one accelerator processor, and the artificial intelligence computing process includes: accessing write-intensive instructions in the primary storage device using the primary processor to perform a setup process; loading read-intensive and non-read-intensive data from the secondary storage device to the at least one accelerator processor, the read-intensive and non-read-intensive data including artificial intelligence model data; and executing a plurality of layer calculations in the artificial intelligence computing process using the at least one accelerator processor based on the artificial intelligence model data, and accessing cache data in the primary storage device via the primary processor; The setup process includes: defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor; inputting data and pre-processing said data; and importing preprocessed data into the plurality of layers of calculation.

7. 7. The artificial intelligence computing process of claim 6, further comprising the step of using a storage controller to determine whether the at least one accelerator processor accesses the secondary storage device.

8. 7. The artificial intelligence computing process of claim 6, further comprising the step of providing an accelerator storage device configured to be connected to the at least one accelerator processor for access by the at least one accelerator processor.

9. 7. The artificial intelligence computing process of claim 6, wherein the primary processor, the at least one accelerator processor, and the storage controller communicate with each other via a PCIe interface.

10. The setup process executed by the primary processor includes: defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor; loading the preprocessed data from the secondary storage device; and setting iteration parameters.

11. 1. A computer hardware system for executing artificial intelligence computing processes, comprising: a primary storage device configured to store write-intensive data; a primary processor coupled to the primary storage device and configured to execute a setup process in the artificial intelligence computing process; a secondary storage device configured to store read-intensive and non-read-intensive data; at least one accelerator processor configured to read the read-intensive and non-read-intensive data stored in the secondary storage device, perform multiple tier calculations in the artificial intelligence computing process based on the read-intensive and non-read-intensive data, and access the write-intensive data in the primary storage device via the primary processor; The setup process executed by the primary processor includes: defining the plurality of layer computations and assigning the plurality of layer computations to the at least one accelerator processor; inputting data and pre-processing said data; and importing the preprocessed data into the plurality of layer calculations.

12. 12. The computer hardware system of claim 11, wherein the write-intensive data includes instructions and cache data for executing the artificial intelligence computing process.

13. 12. The computer hardware system of claim 11, wherein the read-intensive and non-read-intensive data includes artificial intelligence model data.

14. 12. The computer hardware system of claim 11, further comprising a storage controller coupled to the secondary storage device and configured to determine whether the at least one accelerator processor accesses the secondary storage device.

15. 12. The computer hardware system of claim 11, further comprising an accelerator storage device configured to be coupled to the at least one accelerator processor for access by the at least one accelerator processor.

16. 15. The computer hardware system of claim 14, wherein the primary processor, the at least one accelerator processor, and the storage controller communicate with each other via a PCIe interface.

Images