Intelligent calculation system applied to AI large model

Abstract

The invention relates to the technical field of data processing, in particular to an intelligent calculation system applied to an AI large model. The system comprises a computing node cluster, an identification module, a storage analysis module and a scheduling management module. A plurality of computing nodes are deployed to form a computing node cluster, a computing task and control logic are executed through a high-performance CPU, parallel computing and deep learning model training acceleration are performed through a high-performance GPU, rapid data access is provided through a high-bandwidth memory, data transmission delay is reduced, data are collected in real time through an identification module, and the real-time performance of the system is improved. The intelligent computing cluster expansion capability of the computing node cluster is analyzed, the AI large model training process is monitored in real time, the situation that the speed-up ratio is lower than the ideal speed-up ratio in the training process is analyzed, and the intelligent computing cluster expansion capability of the computing node cluster is analyzed by optimizing the storage read-write pressure, the cache mechanism and the dynamic scheduling management of the storage module. Therefore, the approximately linear large-scale cluster expansion capability is improved, and the requirement of large-scale AI model training is met.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an intelligent computing system applied to a large AI model. Background Art

[0002] With the rapid development of artificial intelligence technology, the scale and complexity of large AI models continue to increase, and the demand for computing resources has also skyrocketed. Traditional computing architectures can no longer meet the demand. Therefore, there is an urgent need for an efficient, flexible, and scalable intelligent computing system based on distributed heterogeneous computing and software and hardware collaborative optimization to support the training and reasoning of large AI models. It aims to provide a high-performance, low-power computing environment for large AI models, and have scalability to meet the needs of larger-scale models in the future.

[0003] The patent document with Chinese patent authorization announcement number CN117539726B discloses a green intelligent computing center energy efficiency optimization method and system, which includes the following steps: based on the historical energy consumption records of the intelligent computing center, using association rule mining and clustering analysis algorithms to analyze energy consumption patterns and generate energy consumption analysis reports; it can be seen that when the existing intelligent computing system is applied to the training of large AI models, whenever the number of computing nodes is increased, it is difficult to dynamically adjust the allocation of computing resources through real-time monitoring and dynamic scheduling management, and it is difficult to adapt to the needs of large-scale AI model training. Summary of the invention

[0004] To this end, the present invention provides an intelligent computing system applied to AI large models, so as to overcome the problem in the prior art of low adaptability of the intelligent computing system in AI large model training due to the lack of real-time monitoring of the large model training process and dynamic adjustment of the allocation of computing resources.

[0005] To achieve the above object, the present invention provides an intelligent computing system applied to an AI large model, comprising:

[0006] A computing node cluster, which includes a plurality of computing nodes, any of which includes a high-performance CPU, a high-performance GPU, and a high-bandwidth memory;

[0007] An identification module, which is connected to each of the computing nodes, and is used to monitor the model training process of each high-performance GPU in real time, so as to record the time for the high-performance GPU to complete a single model training, obtain the actual parallel execution time, and calculate the real-time acceleration ratio according to the actual parallel execution time, so as to determine the real-time acceleration ratio according to the standard acceleration ratio, so as to analyze whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard;

[0008] A storage analysis module, for real-time monitoring of the next model training process to determine whether to cache frequently accessed data when it is determined that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard;

[0009] The scheduling management module calculates the real-time memory access density according to the real-time cache strength, so as to dynamically adjust the allocation of computing power resources according to the memory access density.

[0010] Furthermore, the identification module includes a calculation unit, a comparison unit and an analysis unit, wherein:

[0011] The computing unit is used to record the actual parallel execution time of the high-performance GPU when performing a single model training, obtain the number of the computing nodes, and calculate the real-time acceleration ratio according to the actual execution time and the number of the computing nodes;

[0012] The comparison unit is used to compare the standard acceleration ratio with the real-time acceleration ratio to obtain a comparison result;

[0013] The analysis unit is used to analyze the intelligent computing cluster expansion capability of the computing node cluster according to the comparison result.

[0014] Furthermore, the calculation unit includes a pre-calculation subunit, a real-time monitoring subunit and a speedup ratio calculation subunit, wherein:

[0015] The pre-calculation subunit is used to obtain the time taken by a high-performance GPU corresponding to any computing node to complete a single model training, and obtain a single execution time;

[0016] The real-time monitoring subunit is used to obtain the number of the computing nodes and the time taken by the high-performance GPU corresponding to each computing node to complete a single model training, so as to obtain the actual parallel execution time;

[0017] The acceleration ratio calculation subunit is used to calculate the quotient of a single execution time and the product of the actual parallel execution time and the number of computing nodes to obtain a real-time acceleration ratio.

[0018] Further, the analysis unit includes a first analysis subunit and a second analysis subunit, wherein:

[0019] The first analysis subunit is used to determine that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard when the real-time acceleration ratio is less than the standard acceleration ratio;

[0020] The second analysis subunit is used to determine whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard when the real-time acceleration ratio is greater than or equal to the standard acceleration ratio.

[0021] Furthermore, the storage analysis module includes a storage monitoring unit, a storage pressure analysis unit and a cache unit, wherein:

[0022] The storage monitoring unit is used to monitor the read and write operations of each of the computing nodes in real time to obtain the real-time storage delay corresponding to each of the computing nodes;

[0023] The storage pressure analysis unit is used to determine the real-time storage delay according to the allowed storage delay, and to analyze the real-time storage read and write status of each computing node according to the determination result to obtain the corresponding real-time storage read and write pressure status;

[0024] The cache unit is used to obtain the corresponding computing node when the real-time storage read-write pressure is greater than the standard storage read-write pressure, obtain the node to be cached, obtain the data at the node to be cached as frequently accessed data, and analyze the historical data at the node to be cached to cache the frequently accessed data.

[0025] Furthermore, the storage pressure analysis unit includes a determination subunit and a storage analysis subunit, wherein:

[0026] The determination subunit is used to determine the real-time storage delay according to the allowed storage delay;

[0027] The storage analysis subunit is used to obtain the determination result of each of the computing nodes to count the number of the nodes to be cached, and obtain the cache node ratio according to the percentage of the number of nodes to be cached to the total number of computing nodes.

[0028] Further, when the real-time storage delay is less than or equal to the allowed storage delay, the determination subunit obtains a determination result that the corresponding real-time storage read / write pressure condition is normal;

[0029] When the real-time storage delay is greater than the allowed storage delay, a determination result is obtained that the corresponding real-time storage read-write pressure condition is abnormal.

[0030] Furthermore, the cache unit includes a history database, a uniform distribution subunit and a cache subunit, wherein:

[0031] The historical database is used to store the real-time access frequency of each access data;

[0032] The uniform distribution subunit is used to uniformly distribute the access data of the node to be cached to adjacent computing nodes;

[0033] The cache subunit is used to obtain the real-time access frequency corresponding to each access data in the historical database, and cache the frequently accessed data in the order of the real-time access frequency from high to low.

[0034] Furthermore, the scheduling management module includes a memory access intensity calculation unit, wherein:

[0035] A memory access density calculation unit, used to calculate the real-time memory access density according to the real-time cache strength and cache hit rate;

[0036] The real-time cache strength is the product of the cache node ratio and the standard cache strength;

[0037] The cache hit rate is the ratio of the number of read and write requests to the total number of read and write requests;

[0038] The real-time memory access intensity is the product of the real-time cache intensity and the cache hit rate.

[0039] Furthermore, the scheduling management module also includes a resource allocation unit, wherein:

[0040] The resource allocation unit is used to determine the real-time memory access density according to the standard memory access density. When the real-time memory access density is greater than the standard memory access density, the resource allocation unit obtains the real-time storage delay corresponding to each of the computing nodes, and obtains the corresponding computing node sequence in order of the real-time storage delay from high to low, so as to allocate resources in sequence.

[0041] Compared with the prior art, the beneficial effect of the present invention lies in that a number of computing nodes are deployed to form a computing node cluster, and a high-performance CPU, a GPU and a high-bandwidth memory are configured to execute computing tasks and control logic through a high-performance CPU, and parallel computing and accelerated deep learning model training are performed through a high-performance GPU, and fast data access is provided through a high-bandwidth memory to reduce data transmission delay, and data is collected in real time by setting an identification module to analyze the intelligent computing cluster expansion capability of the computing node cluster. In the intelligent computing system composed of the intelligent computing cluster, when the number of processors (computing nodes) is increased, the computing performance can be improved at a nearly linear ratio, that is, if the number of processors is increased by N times, the computing efficiency should also be improved by N times in theory. However, due to the network communication overhead such as parameter synchronization and network communication delay in the actual model training process, the real-time acceleration ratio is lower than the theoretical acceleration ratio. Therefore, by real-time monitoring of the AI ​​large model training process and analyzing the situation where the acceleration ratio is lower than the ideal acceleration ratio during the AI ​​large model training process, the storage read and write pressure, cache mechanism and dynamic scheduling management of the storage module are optimized to improve the approximately linear ultra-large-scale cluster expansion capability and reduce the performance loss caused by network communication overhead.

[0042] Furthermore, the recognition module monitors the performance of each GPU during the model training process in real time, records the time of each model training, calculates the actual parallel execution time, and calculates the real-time acceleration ratio based on this. The real-time acceleration ratio is compared with the standard acceleration ratio to evaluate whether the scalability of the computing node cluster meets the expected standards.

[0043] Furthermore, through the collaborative work of the pre-computation sub-unit and the real-time monitoring sub-unit, the execution time of the GPU of each computing node in the model training can be obtained in real time to promptly reflect the changes in computing performance, facilitate rapid adjustment and optimization of computing resource allocation, and ensure the effective use of computing resources.

[0044] Furthermore, by setting up an analysis unit, the first and second analysis sub-units of the analysis unit can quickly determine whether the expansion capability of the computing node cluster meets the expected standards based on the comparison of the real-time acceleration ratio with the standard acceleration ratio, so that the system can maintain high efficiency during the expansion process and avoid waste of resources.

[0045] Furthermore, the storage monitoring unit monitors the read and write operations of each computing node in real time, obtains real-time storage latency, and promptly identifies changes in storage performance. When the storage read and write pressure exceeds the standard, the cache unit automatically identifies the nodes to be cached, and obtains frequently accessed data for caching through analysis to increase data access speed and reduce storage latency, thereby improving the overall system performance and meeting the needs of AI model training scenarios. By monitoring and analyzing the storage status, it can effectively optimize the utilization of storage resources, avoid unnecessary waste of resources, and improve the overall efficiency of the system. The storage analysis module improves the system's storage performance and resource utilization efficiency through mechanisms such as real-time monitoring, precise analysis, and dynamic caching, providing strong support for efficient computing tasks.

[0046] Furthermore, by determining the real-time storage delay according to the allowed storage delay, changes in storage performance can be identified in a timely manner, by obtaining the determination results of each computing node and counting the number of nodes to be cached, the distribution of storage pressure can be analyzed, and by calculating the proportion of cache nodes, the load of the current storage system can be evaluated. When the real-time storage delay is less than or equal to the allowed storage delay, it is determined that the storage read and write pressure is normal, otherwise it is abnormal, so as to promptly discover and handle high load situations in a timely manner to maintain the stability of the system and adapt to the needs of large-scale computing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a connection diagram of an intelligent computing system applied to an AI large model according to an embodiment of the present invention;

[0048] Figure 2 This is a connection diagram of an identification module according to an embodiment of the present invention;

[0049] Figure 3 A connection diagram of a computing unit according to an embodiment of the present invention;

[0050] Figure 4 A connection diagram of a storage analysis module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0053] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0054] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0055] See also Figure 1 As shown, it is a connection diagram of an intelligent computing system applied to an AI large model according to an embodiment of the present invention. The present invention provides an intelligent computing system applied to an AI large model, including:

[0056] A computing node cluster, which includes a plurality of computing nodes, any of which includes a high-performance CPU, a high-performance GPU, and a high-bandwidth memory;

[0057] An identification module, which is connected to each of the computing nodes, and is used to monitor the model training process of each high-performance GPU in real time, so as to record the time for the high-performance GPU to complete a single model training, obtain the actual parallel execution time, and calculate the real-time acceleration ratio according to the actual parallel execution time, so as to determine the real-time acceleration ratio according to the standard acceleration ratio, so as to analyze whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard;

[0058] A storage analysis module, for real-time monitoring of the next model training process to determine whether to cache frequently accessed data when it is determined that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard;

[0059] The scheduling management module calculates the real-time memory access density according to the real-time cache strength, so as to dynamically adjust the allocation of computing power resources according to the memory access density.

[0060] In this embodiment, a number of computing nodes are deployed to form a computing node cluster. A high-performance CPU, GPU and high-bandwidth memory are configured to execute computing tasks and control logic through a high-performance CPU, perform parallel computing and accelerate deep learning model training through a high-performance GPU, and provide fast data access through a high-bandwidth memory to reduce data transmission delay. The recognition module is set to collect data in real time to analyze the intelligent computing cluster expansion capability of the computing node cluster. In the intelligent computing system composed of the intelligent computing cluster, when the number of processors (computing nodes) is increased, the computing performance can be improved at a nearly linear ratio. That is, if the number of processors is increased by N times, the computing efficiency should also be improved by N times in theory. However, due to the network communication overhead such as parameter synchronization and network communication delay in the actual model training process, the real-time acceleration ratio is lower than the theoretical acceleration ratio. Therefore, by real-time monitoring of the AI ​​large model training process and analyzing the situation where the acceleration ratio is lower than the ideal acceleration ratio during the AI ​​large model training process, the storage read and write pressure, cache mechanism and dynamic scheduling management of the storage module are optimized to improve the approximately linear ultra-large-scale cluster expansion capability and reduce the performance loss caused by network communication overhead.

[0061] See also Figure 2 As shown, it is a connection diagram of the identification module according to an embodiment of the present invention;

[0062] Specifically, the identification module includes a calculation unit, a comparison unit and an analysis unit, wherein:

[0063] The computing unit is used to record the actual parallel execution time of the high-performance GPU when performing a single model training, obtain the number of the computing nodes, and calculate the real-time acceleration ratio according to the actual execution time and the number of the computing nodes;

[0064] The comparison unit is used to compare the standard acceleration ratio with the real-time acceleration ratio to obtain a comparison result;

[0065] The analysis unit is used to analyze the intelligent computing cluster expansion capability of the computing node cluster according to the comparison result.

[0066] The standard speedup ratio set in this embodiment represents the expected approximately linear expansion capability of the intelligent computing cluster by increasing the number of processors in actual model training. The set value is in the range of 0.7-0.95, and the theoretical speedup ratio is expected. When the number of computing nodes increases, the improvement in computing efficiency should be proportional to the number of computing nodes. The theoretical speedup ratio is 1. For example, if a training task of an image recognition model is being monitored, if one processor (computing node) is used to execute it, the task can theoretically be completed in 1 hour. Then, if 10 processors are used for parallel computing, theoretically, if a linear speedup ratio can be achieved, the task can be completed in 1 hour. The completion time should be: theoretical parallel execution time = (1 / 10)h = 0.1h, the theoretical speedup ratio is the quotient of a single execution time and the theoretical parallel execution time, the theoretical speedup ratio = 1h / (10×0.1)h = 1. However, in actual operation, due to the overhead of parameter synchronization and network communication, suppose that in actual operation, the time required for 10 processors to work together is greater than 0.1h, that is, the real-time parallel execution time is greater than the theoretical parallel execution time. For example, the time required for 10 processors to work together is 0.15h, that is, the actual parallel execution time is 0.15 hours, then, the real-time speedup ratio = 1h / (10×0.15)≈0.67.

[0067] The recognition module monitors the performance of each GPU during the model training process in real time, records the time of each model training, calculates the actual parallel execution time, and calculates the real-time acceleration ratio based on this. The real-time acceleration ratio is compared with the standard acceleration ratio to evaluate whether the expansion capability of the computing node cluster meets the expected standards.

[0068] See also Figure 3 As shown, it is a connection diagram of a computing unit according to an embodiment of the present invention;

[0069] Specifically, the calculation unit includes a pre-calculation subunit, a real-time monitoring subunit and an acceleration ratio calculation subunit, wherein:

[0070] The pre-calculation subunit is used to obtain the time taken by a high-performance GPU corresponding to any computing node to complete a single model training, and obtain a single execution time;

[0071] The real-time monitoring subunit is used to obtain the number of the computing nodes and the time taken by the high-performance GPU corresponding to each computing node to complete a single model training, so as to obtain the actual parallel execution time;

[0072] The acceleration ratio calculation subunit is used to calculate the quotient of a single execution time and the product of the actual parallel execution time and the number of computing nodes to obtain a real-time acceleration ratio.

[0073] Through the collaborative work of the pre-computation sub-unit and the real-time monitoring sub-unit, the execution time of the GPU of each computing node in model training can be obtained in real time to promptly reflect changes in computing performance, facilitate rapid adjustment and optimization of computing resource allocation, and ensure effective use of computing resources.

[0074] Specifically, the analysis unit includes a first analysis subunit and a second analysis subunit, wherein:

[0075] The first analysis subunit is used to determine that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard when the real-time acceleration ratio is less than the standard acceleration ratio;

[0076] The second analysis subunit is used to determine whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard when the real-time acceleration ratio is greater than or equal to the standard acceleration ratio.

[0077] By setting up an analysis unit, the first and second analysis sub-units of the analysis unit can quickly determine whether the expansion capability of the computing node cluster meets the expected standard based on the comparison between the real-time acceleration ratio and the standard acceleration ratio, so that the system can maintain high efficiency during the expansion process and avoid waste of resources.

[0078] See also Figure 4 As shown, it is a connection diagram of the storage analysis module according to an embodiment of the present invention;

[0079] Specifically, the storage analysis module includes a storage monitoring unit, a storage pressure analysis unit and a cache unit, wherein:

[0080] The storage monitoring unit is used to monitor the read and write operations of each of the computing nodes in real time to obtain the real-time storage delay corresponding to each of the computing nodes;

[0081] The storage pressure analysis unit is used to determine the real-time storage delay according to the allowed storage delay, and to analyze the real-time storage read and write status of each computing node according to the determination result to obtain the corresponding real-time storage read and write pressure status;

[0082] The cache unit is used to obtain the corresponding computing node when the real-time storage read-write pressure is greater than the standard storage read-write pressure, obtain the node to be cached, obtain the data at the node to be cached as frequently accessed data, and analyze the historical data at the node to be cached to cache the frequently accessed data.

[0083] The standard storage read / write pressure set in this embodiment is the maximum storage read / write load that the system can withstand. The set value is related to the design capability and performance index of the system and is used to evaluate the performance of the storage system under high load. The standard storage read / write pressure can be set to 1000 IOPS (input / output operations per second), which means that under normal operation, the maximum read / write request that the node can handle is 1000 times per second. The allowed storage delay means the maximum delay time that the system can accept when performing storage read / write operations. If this time is exceeded, the storage performance is judged to be substandard, and the corresponding optimization or adjustment measures are triggered. The allowed storage delay is set to 5 milliseconds (ms). When performing read / write operations, any delay exceeding 5 milliseconds will be judged as abnormal, which may affect the performance of the system.

[0084] The storage monitoring unit monitors the read and write operations of each computing node in real time, obtains real-time storage latency, and promptly identifies changes in storage performance. When the storage read and write pressure exceeds the standard, the cache unit automatically identifies the nodes to be cached, and obtains frequently accessed data for caching through analysis to increase data access speed and reduce storage latency, thereby improving overall system performance and meeting the needs of AI model training scenarios. By monitoring and analyzing the storage status, it can effectively optimize the utilization of storage resources, avoid unnecessary waste of resources, and improve the overall efficiency of the system. The storage analysis module improves the system's storage performance and resource utilization efficiency through mechanisms such as real-time monitoring, precise analysis, and dynamic caching, providing strong support for efficient computing tasks.

[0085] Specifically, the storage pressure analysis unit includes a determination subunit and a storage analysis subunit, wherein:

[0086] The determination subunit is used to determine the real-time storage delay according to the allowed storage delay;

[0087] The storage analysis subunit is used to obtain the determination result of each of the computing nodes to count the number of the nodes to be cached, and obtain the cache node ratio according to the percentage of the number of nodes to be cached to the total number of computing nodes;

[0088] Among them, when the real-time storage delay is less than or equal to the allowed storage delay, the judgment result is that the corresponding real-time storage read-write pressure condition is normal; when the real-time storage delay is greater than the allowed storage delay, the judgment result is that the corresponding real-time storage read-write pressure condition is abnormal.

[0089] By judging the real-time storage delay according to the allowed storage delay, the changes in storage performance can be identified in time. By obtaining the judgment results of each computing node and counting the number of nodes to be cached, the distribution of storage pressure can be analyzed. By calculating the proportion of cache nodes, the load of the current storage system can be evaluated. When the real-time storage delay is less than or equal to the allowed storage delay, it is judged that the storage read and write pressure is normal, otherwise it is abnormal. In this way, high load situations can be discovered and handled in time to maintain system stability and adapt to the needs of large-scale computing tasks.

[0090] Specifically, the cache unit includes a history database, a uniform distribution subunit and a cache subunit, wherein:

[0091] The historical database is used to store the real-time access frequency of each access data;

[0092] The uniform distribution subunit is used to uniformly distribute the access data of the node to be cached to adjacent computing nodes;

[0093] The cache subunit is used to obtain the real-time access frequency corresponding to each access data in the historical database, and cache the frequently accessed data in the order of the real-time access frequency from high to low.

[0094] The access frequency of each data item is recorded through the historical database, and the access data of the node to be cached is evenly distributed to the adjacent computing nodes through the uniform distribution sub-unit to balance the load of each computing node. The frequently accessed data is cached in the order of real-time access frequency from high to low through the cache sub-unit, so that the least frequently used or least recently used data will not be cached, so that the most frequently used data can be quickly accessed, thereby improving the overall performance of the system.

[0095] Specifically, the scheduling management module includes a memory access intensity calculation unit, wherein:

[0096] A memory access density calculation unit, used to calculate the real-time memory access density according to the real-time cache strength and cache hit rate;

[0097] The real-time cache strength is the product of the cache node ratio and the standard cache strength;

[0098] The cache hit rate is the ratio of the number of read and write requests to the total number of read and write requests;

[0099] The real-time memory access intensity is the product of the real-time cache intensity and the cache hit rate.

[0100] In this embodiment, the number of read and write requests is the number of data requests successfully obtained from the cache, the total number of read and write requests is the number of all read and write requests issued within a preset time period, the preset time period is the monitoring duration set in the model training, which is set to 10ms, and the standard cache strength represents the strength threshold of the cache in the system, and the standard cache strength is set to 400MB.

[0101] Specifically, the scheduling management module also includes a resource allocation unit, wherein:

[0102] The resource allocation unit is used to determine the real-time memory access density according to the standard memory access density. When the real-time memory access density is greater than the standard memory access density, the resource allocation unit obtains the real-time storage delay corresponding to each of the computing nodes, and obtains the corresponding computing node sequence in descending order of the real-time storage delay, so as to sequentially allocate resources;

[0103] When the real-time memory access density is less than or equal to the standard memory access density, the resource allocation unit does not adjust the allocation state of the computing power resources.

[0104] The standard memory access density set in this embodiment represents the maximum memory access load of the system, which is a value set based on the system's performance goals and historical operation data. After historical data analysis and performance testing, the set standard memory access density is 150MB. For example, the total number of read and write requests is 1000, the number of hit requests is 800, the cache node ratio is 50%, the real-time cache strength is 200MB, and the calculated real-time memory access density is 160MB. At this time, the real-time memory access density (160MB) is greater than the standard memory access density (150MB), so the resource allocation unit will adjust resources according to the real-time storage delay to optimize system performance.

[0105] The resource allocation unit obtains the corresponding computing node sequence in the order of real-time storage latency from high to low, and allocates resources in sequence. That is, by allocating tasks with high memory access requirements to nodes with higher latency, the memory access time is reduced and the task execution efficiency is improved. For tasks with low memory access requirements, there is no need to adjust resource allocation to avoid resource waste. According to the actual computing power requirements, the resource allocation strategy is dynamically adjusted to optimize memory access performance and improve the overall system efficiency and resource utilization.

[0106] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent computing system applied to a large AI model, characterized in that: include, A computing node cluster, which includes a plurality of computing nodes, any of which includes a high-performance CPU, a high-performance GPU, and a high-bandwidth memory; An identification module, which is connected to each of the computing nodes, and is used to monitor the model training process of each high-performance GPU in real time, so as to record the time for the high-performance GPU to complete a single model training, obtain the actual parallel execution time, and calculate the real-time acceleration ratio according to the actual parallel execution time, so as to determine the real-time acceleration ratio according to the standard acceleration ratio, so as to analyze whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard; A storage analysis module, for real-time monitoring of the next model training process to determine whether to cache frequently accessed data when it is determined that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard; The scheduling management module calculates the real-time memory access density according to the real-time cache strength, so as to dynamically adjust the allocation of computing power resources according to the memory access density.

2. The intelligent computing system applied to the AI ​​large model according to claim 1 is characterized in that: The identification module includes a calculation unit, a comparison unit and an analysis unit, wherein: The computing unit is used to record the actual parallel execution time of the high-performance GPU when performing a single model training, obtain the number of the computing nodes, and calculate the real-time acceleration ratio according to the actual execution time and the number of the computing nodes; The comparison unit is used to compare the standard acceleration ratio with the real-time acceleration ratio to obtain a comparison result; The analysis unit is used to analyze the intelligent computing cluster expansion capability of the computing node cluster according to the comparison result.

3. The intelligent computing system applied to the AI ​​large model according to claim 2 is characterized in that: The calculation unit includes a pre-calculation subunit, a real-time monitoring subunit and an acceleration ratio calculation subunit, wherein: The pre-calculation subunit is used to obtain the time taken by a high-performance GPU corresponding to any computing node to complete a single model training, and obtain a single execution time; The real-time monitoring subunit is used to obtain the number of the computing nodes and the time taken by the high-performance GPU corresponding to each computing node to complete a single model training, so as to obtain the actual parallel execution time; The acceleration ratio calculation subunit is used to calculate the quotient of a single execution time and the product of the actual parallel execution time and the number of computing nodes to obtain a real-time acceleration ratio.

4. The intelligent computing system applied to the AI ​​large model according to claim 2 is characterized in that: The analysis unit comprises a first analysis subunit and a second analysis subunit, wherein: The first analysis subunit is used to determine that the intelligent computing cluster expansion capability of the computing node cluster does not meet the standard when the real-time acceleration ratio is less than the standard acceleration ratio; The second analysis subunit is used to determine whether the intelligent computing cluster expansion capability of the computing node cluster meets the standard when the real-time acceleration ratio is greater than or equal to the standard acceleration ratio.

5. The intelligent computing system applied to the AI ​​large model according to claim 1 is characterized in that: The storage analysis module includes a storage monitoring unit, a storage pressure analysis unit and a cache unit, wherein: The storage monitoring unit is used to monitor the read and write operations of each of the computing nodes in real time to obtain the real-time storage delay corresponding to each of the computing nodes; The storage pressure analysis unit is used to determine the real-time storage delay according to the allowed storage delay, and to analyze the real-time storage read and write status of each computing node according to the determination result to obtain the corresponding real-time storage read and write pressure status; The cache unit is used to obtain the corresponding computing node when the real-time storage read-write pressure is greater than the standard storage read-write pressure, obtain the node to be cached, obtain the data at the node to be cached as frequently accessed data, and analyze the historical data at the node to be cached to cache the frequently accessed data.

6. The intelligent computing system applied to the AI ​​large model according to claim 5 is characterized in that: The storage pressure analysis unit includes a determination subunit and a storage analysis subunit, wherein: The determination subunit is used to determine the real-time storage delay according to the allowed storage delay; The storage analysis subunit is used to obtain the determination result of each of the computing nodes to count the number of the nodes to be cached, and obtain the cache node ratio according to the percentage of the number of nodes to be cached to the total number of computing nodes.

7. The intelligent computing system applied to the AI ​​large model according to claim 6 is characterized in that: When the real-time storage delay is less than or equal to the allowed storage delay, the determination subunit determines that the corresponding real-time storage read / write pressure condition is normal; When the real-time storage delay is greater than the allowed storage delay, a determination result is obtained that the corresponding real-time storage read-write pressure condition is abnormal.

8. The intelligent computing system applied to the AI ​​large model according to claim 5 is characterized in that: The cache unit includes a history database, a uniform distribution subunit and a cache subunit, wherein: The historical database is used to store the real-time access frequency of each access data; The uniform distribution subunit is used to uniformly distribute the access data of the node to be cached to adjacent computing nodes; The cache subunit is used to obtain the real-time access frequency corresponding to each access data in the historical database, and cache the frequently accessed data in the order of the real-time access frequency from high to low.

9. The intelligent computing system applied to the AI ​​large model according to claim 1 is characterized in that: The scheduling management module includes a memory access intensity calculation unit, wherein: A memory access density calculation unit, used to calculate the real-time memory access density according to the real-time cache strength and cache hit rate; The real-time cache strength is the product of the cache node ratio and the standard cache strength; The cache hit rate is the ratio of the number of read and write requests to the total number of read and write requests; The real-time memory access intensity is the product of the real-time cache intensity and the cache hit rate.

10. The intelligent computing system applied to the AI ​​large model according to claim 9, characterized in that: The scheduling management module also includes a resource allocation unit, wherein: The resource allocation unit is used to determine the real-time memory access density according to the standard memory access density. When the real-time memory access density is greater than the standard memory access density, the resource allocation unit obtains the real-time storage delay corresponding to each of the computing nodes, and obtains the corresponding computing node sequence in order of the real-time storage delay from high to low, so as to allocate resources in sequence.

Images