Task processing method, device, equipment and storage medium based on intelligent computing cluster

Abstract

The embodiments of the present application provide a task processing method, apparatus, device and storage medium based on an intelligent computing cluster. The method includes: obtaining a task to be processed and determining the corresponding intelligent computing cluster; determining multiple first processors and multiple second processors from the intelligent computing cluster, wherein the first operating frequency corresponding to each first processor is greater than the second operating frequency corresponding to each second processor; pre-filling the task to be processed by the multiple first processors corresponding to the first operating frequency to obtain intermediate features, and transmitting the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor; when an update to the intermediate data temporary storage area is detected, the intermediate features are obtained from the intermediate data temporary storage area and decoded by the multiple second processors corresponding to the second operating frequency to obtain the decoding results of each second processor; and the decoding results of each second processor are output in sequence. In this way, energy consumption can be saved at the source.

Description

Technical Field

[0001] The present application relates to the field of resource scheduling, and in particular to a task processing method, apparatus, device and storage medium based on an intelligent computing cluster. Background Art

[0002] An intelligent computing cluster is a computing resource and architecture that combines high-performance computing capabilities with intelligent technologies such as artificial intelligence and big data analytics. It aims to improve computing resource utilization and accelerate the execution of complex computing tasks through intelligent management and optimization strategies. However, running AI models on an intelligent computing cluster is a major source of energy consumption. Taking the GPT3 model as an example, its training process consumes approximately 1287MWh of electricity, equivalent to the energy consumed by a medium-sized data center running continuously for several months. Therefore, it is necessary to propose methods to save energy when running AI models on intelligent computing clusters.

[0003] In related technologies, to reduce processor idle time and eliminate resource waste, multiple copies of the same AI model can be deployed in a computing cluster to achieve parallel processing of tasks, minimize processor waiting time, and avoid energy waste. However, while this approach alleviates the energy waste problem to a certain extent, it cannot reduce the energy consumption of the processor when processing tasks, and cannot save energy at the root cause. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a task processing method, device, equipment and storage medium based on an intelligent computing cluster, which can save energy consumption at the root.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a task processing method based on an intelligent computing cluster, the method comprising:

[0006] Obtaining pending tasks and determining an intelligent computing cluster for processing the pending tasks;

[0007] Determine, according to a preset partitioning rule, a plurality of first processors and a plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency;

[0008] Pre-filling the to-be-processed task by the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmitting the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor;

[0009] When an update of the intermediate data temporary storage area is detected, the intermediate features are obtained from the intermediate data temporary storage area by the multiple second processors corresponding to the second operating frequency, and the intermediate features are decoded to obtain a decoding result of each second processor;

[0010] The decoding results of each second processor are outputted in sequence.

[0011] Accordingly, a second aspect of an embodiment of the present application proposes a task processing device based on an intelligent computing cluster, the device comprising:

[0012] An acquisition module is used to acquire tasks to be processed and determine an intelligent computing cluster for processing the tasks to be processed;

[0013] a determination module, configured to determine, according to a preset partitioning rule, a plurality of first processors and a plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency;

[0014] a processing module, configured to perform pre-filling processing on the to-be-processed task by using the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmit the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor;

[0015] a decoding module configured to, when detecting that the intermediate data temporary storage area is updated, obtain the intermediate features from the intermediate data temporary storage area through the plurality of second processors corresponding to the second operating frequency, and decode the intermediate features to obtain a decoding result of each second processor;

[0016] An output module is used to sequentially output the decoding results of each second processor.

[0017] In some implementations, the first processor and the second processor are located in the same cluster node; and the determining module is further configured to:

[0018] Determining a cluster node for processing the task to be processed from the intelligent computing cluster;

[0019] Determining a first division ratio and a second division ratio for the plurality of intelligent computing processors of the cluster node according to a preset division rule;

[0020] Determining a plurality of first processors from the plurality of intelligent computing processors according to the first division ratio;

[0021] According to the second division ratio, a plurality of second processors are determined from the plurality of intelligent computing processors, and the total number of the plurality of first processors and the plurality of second processors is equal to the total number of intelligent computing processors of the cluster node.

[0022] In some embodiments, the task processing device based on the intelligent computing cluster further includes an adjustment module for:

[0023] In the process of processing the tasks to be processed, regularly obtaining the average delay of the first characters of the pre-filling processing of the tasks to be processed by the multiple first processors and the average delay of the second characters of the decoding processing of the tasks to be processed by the multiple second processors;

[0024] Comparing the first character average delay with the second character average delay to obtain a first comparison result;

[0025] When the first comparison result indicates that the first character average delay is inconsistent with the second character average delay, the number of the plurality of first processors and the number of the plurality of second processors are adjusted according to the first comparison result.

[0026] In some embodiments, the adjustment module is further configured to:

[0027] When the first comparison result indicates that the average delay of the first character is greater than the average delay of the second character, obtaining a first difference between the average delay of the first character and the average delay of the second character;

[0028] determining a first number of additional first processors according to a first difference range corresponding to the first difference, and adjusting the number of the plurality of first processors and the plurality of second processors according to the first number;

[0029] When the first comparison result indicates that the average delay of the first character is less than the average delay of the second character, obtaining a second difference between the average delay of the second character and the average delay of the first character;

[0030] A second number of added second processors is determined according to a second difference range corresponding to the second difference, and the number of the plurality of first processors and the plurality of second processors is adjusted according to the second number.

[0031] In some embodiments, the adjustment module is further configured to:

[0032] Obtaining the sum of the average delay of the first character and the average delay of the second character to obtain a total delay;

[0033] Obtaining a preset delay threshold, and comparing the delay threshold with the total delay to obtain a second comparison result;

[0034] When the second comparison result indicates that the total delay is greater than the delay threshold, determining a third operating frequency corresponding to the second processor, and obtaining the intermediate features from the intermediate data temporary storage area by the multiple second processors corresponding to the third operating frequency, and decoding the intermediate features to obtain a decoding result of each second processor; wherein the third operating frequency is greater than the second operating frequency;

[0035] When the second comparison result indicates that the total delay is less than the delay threshold, the fourth operating frequency corresponding to the second processor is determined, and the intermediate features are obtained from the intermediate data temporary area through the multiple second processors corresponding to the fourth operating frequency, and the intermediate features are decoded to obtain the decoding results of each second processor; wherein, the fourth operating frequency is less than the second operating frequency.

[0036] In some embodiments, the task processing device based on the intelligent computing cluster further includes a prediction module for:

[0037] Setting the first operating frequencies corresponding to the plurality of first processors as the highest operating frequency;

[0038] Based on the highest operating frequency, predicting the total operating power consumption and total operating delay of the cluster nodes corresponding to the multiple first processors and the multiple second processors at different operating frequencies of the multiple second processors;

[0039] Drawing a power consumption curve based on the total power consumption of the operation, and drawing a performance curve based on the total delay of the operation;

[0040] Based on the power consumption curve and the performance curve, second operating frequencies of the plurality of second processors are determined.

[0041] In some embodiments, the task processing device based on the intelligent computing cluster further includes a numbering module for:

[0042] Divide the task to be processed into multiple sub-tasks, and number the multiple sub-tasks according to the order of tasks;

[0043] In the intermediate data temporary storage area, the received intermediate features are stored according to the task number of each sub-processing task, and the intermediate features are sequentially obtained from the intermediate data temporary storage area according to the corresponding task numbers through the multiple second processors.

[0044] Correspondingly, the third aspect of the embodiments of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the task processing method based on the intelligent computing cluster described in any one of the embodiments of the first aspect of the present application.

[0045] Correspondingly, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the task processing method based on the intelligent computing cluster described in any one of the embodiments of the first aspect of the present application.

[0046] The embodiment of the present application obtains tasks to be processed and determines an intelligent computing cluster for processing the tasks to be processed; according to a preset division rule, multiple first processors and multiple second processors are determined from multiple intelligent computing processors in the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency; pre-filling processing is performed on the tasks to be processed by the multiple first processors corresponding to the first operating frequency to obtain intermediate features, and the intermediate features are transmitted to an intermediate data temporary storage area shared by the first processor and the second processor; when an update of the intermediate data temporary storage area is detected, the intermediate features are obtained from the intermediate data temporary storage area by the multiple second processors corresponding to the second operating frequency, and the intermediate features are decoded to obtain a decoding result of each second processor; and the decoding result of each second processor is output in sequence. In this way, different intelligent computing processors can be divided for processing according to the sensitivity of the tasks to be processed to the operating frequency of the intelligent computing processor at different processing stages. The first processor executes the pre-filling task and the second processor executes the decoding task. The first operating frequency corresponding to the first processor is set to be greater than the second operating frequency corresponding to the second processor. In this way, the pre-filling task that is more sensitive to frequency can be processed by the first processor with a higher frequency to quickly complete data processing; the decoding task that is less sensitive to frequency can be processed by the second processor with a lower frequency, so as to reduce energy consumption as much as possible while maintaining high system performance, so as to effectively reduce the energy consumption of the processor without affecting performance, thereby saving energy at the root. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a schematic diagram of the hardware architecture of the intelligent computing cluster provided in an embodiment of the present application;

[0048] Figure 2 This is a flowchart of a task processing method based on an intelligent computing cluster provided in an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of the functional modules of a task processing device based on an intelligent computing cluster provided in an embodiment of the present application;

[0050] Figure 4 This is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0054] An intelligent computing cluster is a computing resource and architecture that combines high-performance computing capabilities with intelligent technologies (such as artificial intelligence and big data analytics). It aims to improve computing resource utilization and accelerate the execution of complex computing tasks through intelligent management and optimization strategies. However, running AI models on an intelligent computing cluster is a major source of energy consumption. Taking the GPT3 model as an example, its training process consumes approximately 1287MWh (megawatt-hours) of electricity, equivalent to the energy consumed by a medium-sized data center running continuously for several months. Therefore, it is necessary to propose methods to save energy when running AI models on intelligent computing clusters.

[0055] In related technologies, to reduce processor idle time and eliminate resource waste, multiple copies of the same AI model can be deployed in a computing cluster to achieve parallel processing of tasks, minimize processor waiting time, and avoid energy waste. However, while this approach alleviates the energy waste problem to a certain extent, it cannot reduce the energy consumption of the processor when processing tasks, and cannot save energy at the root cause.

[0056] Based on this, the embodiments of the present application provide a task processing method, apparatus, device and storage medium based on an intelligent computing cluster, which can save energy consumption at the root.

[0057] The task processing method, apparatus, device and storage medium based on the intelligent computing cluster provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the hardware architecture of the intelligent computing cluster in the embodiments of the present application is described.

[0058] Please refer to Figure 1 In some implementations, embodiments of the present application provide a task processing system based on an intelligent computing cluster. The intelligent computing cluster includes multiple intelligent computing processors, which can be used to perform reasoning tasks for large AI models or other complex tasks that require a large amount of computing resources.

[0059] Furthermore, the intelligent computing processor is a unit that performs specific computing tasks. The intelligent computing processor can be divided into a first processor specifically for processing pre-filling tasks and a second processor specifically for processing decoding tasks. Among them, the first processor can be used for pre-filling tasks, that is, processing the pending tasks input by the user through the terminal and converting them into feature representations that can be understood by the AI ​​large model. The first processor runs at a high frequency to ensure that computing-intensive tasks are completed quickly; the second processor can be used for decoding tasks, that is, generating outputs based on the intermediate feature representations obtained in the pre-filling stage. Since decoding tasks are memory-intensive tasks and are less sensitive to processor frequency, the second processor can run at a lower frequency.

[0060] In some implementations, the intelligent computing server can be installed on a computer device, which can be responsible for collecting and managing resource information for all intelligent computing clusters, including cluster node load, available resources, geographic location, and the allocation of intelligent computing processors to each cluster node. The computer device can monitor the latency of pre-filling and decoding tasks and promptly adjust the number of intelligent computing processors used to handle pre-filling and decoding tasks to optimize energy efficiency while ensuring performance.

[0061] The task processing method based on the intelligent computing cluster in the embodiment of the present application can be illustrated by the following embodiment.

[0062] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0063] In the embodiment of the present application, the task processing device based on the intelligent computing cluster will be described from the perspective of the task processing device based on the intelligent computing cluster. The task processing device based on the intelligent computing cluster can be integrated into a computer device. Figure 2 , Figure 2 This is a flowchart of the steps of the task processing method based on the intelligent computing cluster provided in an embodiment of the present application. In this embodiment of the present application, the task processing device based on the intelligent computing cluster is specifically integrated into a terminal or server as an example. When the processor on the terminal or server executes the program instructions corresponding to the task processing method based on the intelligent computing cluster, the specific process is as follows:

[0064] Step 101: Obtain tasks to be processed and determine an intelligent computing cluster for processing the tasks to be processed.

[0065] In some embodiments, in order to understand the user's specific request and ensure efficient completion of tasks, pending tasks can be obtained and an intelligent computing cluster can be determined to process the pending tasks to determine the type and content of the tasks, ensure that the tasks are processed correctly, and improve user experience and service quality.

[0066] Among them, pending tasks can be AI model inference tasks that need to be executed on the intelligent computing cluster. Pending tasks include but are not limited to natural language processing, image recognition, speech recognition, etc. Specifically, pending tasks can be requests initiated by users through terminals, such as translating a text, generating a reply, recognizing an image, training a model, etc.

[0067] An intelligent computing cluster is a high-performance computing platform composed of multiple intelligent computing processors (e.g., AI processors), typically located in a data center or cloud environment, capable of handling large-scale data computing needs. These processors can be graphics processing units (GPUs) or neural processing units (NPUs), and are used to efficiently execute complex AI tasks such as model training and inference.

[0068] For example, a user may initiate a request through a terminal (such as a smartphone or personal computer), which may include the specific content of the task to be processed, such as the text to be translated or the image to be recognized. After the terminal sends the request containing the task to be processed to the computer device via the network, the computer device may parse the content of the task to be processed, such as the task type, input data, and the expected output format.

[0069] Furthermore, the computer device can evaluate the available resources in the intelligent computing cluster and, based on the results of the analysis of the pending task, determine the corresponding cluster nodes and processors to handle the pending task. Alternatively, the computer device can directly determine the intelligent computing cluster with the smallest task load as the intelligent computing cluster to handle the pending task based on the task load of each intelligent computing cluster.

[0070] To reduce processor idle time and eliminate resource waste, multiple copies of the same AI model can be deployed in an intelligent computing cluster to enable parallel processing of tasks. For example, multiple intelligent computing processors can process the same pending task in parallel, minimizing the processor's waiting time while avoiding energy waste.

[0071] Through the above steps, we can effectively obtain tasks to be processed and determine the intelligent computing cluster used to process the tasks, ensuring the orderly distribution of tasks.

[0072] Step 102: According to a preset division rule, determine multiple first processors and multiple second processors from multiple intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency.

[0073] In some embodiments, in order to achieve energy efficiency optimization and performance improvement of task processing, different intelligent computing processors can be used to process different processing stages of the task to be processed according to preset division rules, thereby allocating intelligent computing processors with different operating frequencies according to task characteristics, so as to essentially save energy consumption without affecting the performance of the intelligent computing cluster.

[0074] Among them, the preset division rule can be a rule for dividing the number of intelligent computing processors according to a preset standard or algorithm.

[0075] Among them, the intelligent computing processor can be the computing unit that constitutes the intelligent computing cluster. The intelligent computing processor has high-performance computing capabilities and is suitable for executing complex artificial intelligence tasks, including but not limited to graphics processors, neural network processors, etc.

[0076] The first processor may be a group of intelligent computing processors selected from the intelligent computing cluster according to a preset division rule, and is configured to execute tasks to be processed in the pre-population phase. The first processor is configured to operate at a first operating frequency to maximize its computing efficiency.

[0077] The second processor may be a group of intelligent computing processors selected from the intelligent computing cluster according to a preset division rule, and is configured to execute the pending tasks in the decoding phase. The second processor is configured to operate at a second operating frequency to optimize energy efficiency.

[0078] The first operating frequency may be an operating frequency allocated to the first processor, which may be an upper limit of the maximum operating frequency of the first processor.

[0079] The second operating frequency may be an operating frequency allocated to the second processor, which is lower than the first operating frequency, for example, 75%, 60%, etc. of the first operating frequency.

[0080] In some embodiments, since pre-filling the pending tasks involves a large number of computational tasks, such as matrix multiplication and convolution operations, which place high demands on the computing power of the intelligent computing processor and are computationally intensive tasks, the first processor needs to run at a higher frequency (a first operating frequency) to ensure computational efficiency. However, decoding the intermediate features obtained from the pre-filling process primarily involves a large number of memory access tasks, which places relatively low demands on the computing power of the intelligent computing processor. Therefore, the second processor can run at a lower frequency (a second operating frequency) to reduce power consumption.

[0081] Exemplarily, the preset division rule may be to divide the multiple intelligent computing clusters according to a preset ratio. For example, if the division ratio of the first processor and the second processor is 50%, when a cluster node has 8 intelligent computing processors, 4 of them can be set as the first processor and the other 4 as the second processor.

[0082] In some implementations, an intelligent scheduling algorithm can be used to analyze the characteristics of pending tasks, obtain task characteristic analysis results, and adjust the number of first and second processors based on the task characteristic analysis results. For example, for computationally intensive tasks, the number of first processors can be increased; for memory-intensive tasks, the number of second processors can be increased.

[0083] For example, an intelligent scheduling algorithm can be used to analyze the characteristics of pending tasks, categorize them, and establish a database containing the typical characteristics of each task type. For each task type, the corresponding number of computational operations and memory accesses are recorded. The first and second processors for the current pending task are then divided based on the allocation ratios of the first and second processors for each task type recorded in the database.

[0084] In some embodiments, the number of computing operations when the task to be processed is in the pre-filling stage can also be predicted, and the number of memory accesses when the task to be processed is in the decoding stage can be predicted. Afterwards, the ratio of the first processor to the second processor is calculated based on the number of computing operations and the number of memory accesses. For example, if the number of computing operations is 10,000 when the task to be processed is in the pre-filling stage, and the number of memory accesses is 5,000 when the task to be processed is in the decoding stage, then the ratio of the first processor can be 10,000 / (10,000+5,000)≈2 / 3; the ratio of the second processor can be 5,000 / (10,000+5,000)≈1 / 3. Then, when a cluster node has 8 intelligent computing processors, 5 of them can be set as the first processor and the other 3 as the second processor. Furthermore, the prediction of the number of computing operations when the task to be processed is in the pre-filling stage can be made by a machine learning algorithm (such as a regression model, a neural network, a decision tree, etc.), or it can be made based on historical data. The embodiments of the present application do not impose specific restrictions on this.

[0085] In some implementations, the division ratio of the first processor to the second processor may be manually set, and the multiple intelligent computing processors of the intelligent computing cluster may be divided accordingly, for example, the first processor to the second processor may be set to 4 to 5, and so on.

[0086] By running the pre-filling tasks and decoding tasks corresponding to different stages of the tasks to be processed on different intelligent computing processors and setting different operating frequencies for the two, energy consumption in the decoding stage can be effectively saved without affecting system performance. This is of great significance in large-scale cluster systems and is conducive to environmental sustainable development.

[0087] Step 103 , pre-filling the tasks to be processed by the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmitting the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor.

[0088] In some implementations, in order to improve the efficiency of task processing and the reliability of data, an intermediate buffer area may be set up to centrally manage intermediate features to ensure the orderly transmission and integrity of data.

[0089] Among them, pre-filling processing of the task to be processed can be the process of processing input data (such as text, voice, image, etc.) into feature representations suitable for model understanding.

[0090] Intermediate features can be temporary results or feature representations generated during the pre-population phase of processing pending tasks. Intermediate features are the data format obtained after preliminary processing of input data (such as text, speech, and images). They are designed to be suitable for internal processing within the model to facilitate subsequent decoding tasks.

[0091] The intermediate data temporary storage area can be a storage area located within the intelligent computing cluster node, used to temporarily store intermediate features generated during the pre-filling phase to support efficient data exchange between the pre-filling task and the decoding task. Because the tasks to be processed are executed by the first processor and the second processor at different stages, the intermediate data temporary storage area can be used to ensure that the intermediate features can be seamlessly transferred from the pre-filling task to the decoding task.

[0092] For example, to increase computing speed and reduce latency, the maximum frequency of each first processor can be used as the first operating frequency. For example, if the task to be processed is a large-scale natural language processing task, where the task to be processed includes 100 sentences, each containing 100 words, the first processor can convert the input text into a vector representation within the model at a first operating frequency, such as 1.5 GHz (gigahertz), and generate intermediate features. The generated intermediate features are stored in the intermediate data temporary storage area within the node.

[0093] For example, when the tasks to be processed are processed in the same intelligent computing cluster node, a high-speed cache (such as SRAM, DRAM) can be used as an intermediate data temporary storage area. The intermediate data temporary storage area allows multiple intelligent computing processors to access it simultaneously, providing low-latency data access and reducing the overhead of data copying.

[0094] Furthermore, the intermediate data temporary storage area can be mapped to the address space of each intelligent computing processor (including the first processor and the second processor), so that each intelligent computing processor can directly access the data in the shared intermediate data temporary storage area. Furthermore, multiple first processors can directly write the generated intermediate features into the intermediate data temporary storage area, and multiple second processors can directly read these intermediate features from the intermediate data temporary storage area via the high-speed interconnection within the node.

[0095] In some embodiments, when the task to be processed needs to be processed across cluster nodes, the intermediate data temporary storage area is a storage area located between different cluster nodes, which is used to temporarily store intermediate features that need to be transmitted across nodes. At this time, a distributed storage system (such as a distributed file system, distributed cache) can be used as the intermediate data temporary storage area to provide high availability and scalability.

[0096] In some embodiments, to further improve data processing efficiency and energy efficiency, a multi-level caching strategy can be implemented in the intermediate data temporary storage area shared by the first and second processors. Different levels of cache areas can correspond to storage media with different speeds, thereby maintaining fast data access speeds while reducing the number of accesses to main memory. For example, in addition to the area for storing intermediate features, the intermediate data temporary storage area can be divided into three levels of cache areas. The first level of cache has the fastest access rate but the smallest capacity and is used to store data that is most frequently accessed during the pre-fill task, such as feature representations of recently processed input data. The second level of cache is slower than the first level of cache but has a larger capacity and is used to store data that may be reused during the pre-fill task or data that overflows from the first level of cache. The third level of cache has an even lower access speed than the second level of cache to provide more storage space for data that is less frequently accessed but occasionally needed during the pre-fill process or data that overflows from the second level of cache. It is important to note that data consistency should be maintained between the different levels of cache to avoid data conflicts. In some embodiments, the sizes of the different levels of cache can be dynamically adjusted based on the load and characteristics of the pre-fill task.

[0097] It can be understood that by dividing the intermediate data temporary storage area into three levels of cache areas, the data that may be needed can be predicted and loaded into the cache in advance based on the access pattern of the pre-filled task. Through these hierarchical cache strategies, the pre-filled processor's access requirements to the main memory can be effectively reduced, energy consumption can be reduced, and the speed and efficiency of data processing can be improved.

[0098] By using intermediate data buffers and high-speed interconnections to achieve efficient data transfer, it is possible to ensure that the intermediate features generated by the first processor can be quickly and efficiently transferred to the second processor, thereby significantly improving task processing efficiency, data reliability, and resource utilization, and optimizing the overall performance and energy efficiency of the system.

[0099] By using the high-frequency first processor to process pre-fill tasks, data processing can be completed quickly, task delays can be reduced, and user experience can be improved.

[0100] Step 104 : When it is detected that the intermediate data temporary storage area is updated, the intermediate features are obtained from the intermediate data temporary storage area by using multiple second processors corresponding to the second operating frequency, and the intermediate features are decoded to obtain decoding results of each second processor.

[0101] In some embodiments, in order to ensure that the decoding processor can obtain the latest intermediate features in a timely manner and reduce the delay in data transmission, when an update of the intermediate data temporary storage area is detected, the intermediate features can be obtained from the intermediate data temporary storage area through the second processor corresponding to the second operating frequency, thereby reducing power consumption and improving the energy efficiency of the system.

[0102] Among them, decoding the intermediate features can be a process of generating output based on the intermediate feature representation obtained in the pre-filling stage.

[0103] The decoding result can be the final product of the model inference process, such as text, labels, classification results, or predicted values. The specific form depends on the nature and objectives of the task being processed. For example, in natural language processing tasks, the decoding result may be a translated sentence; in image recognition tasks, the decoding result may be the label of an object in the image.

[0104] For example, to ensure that energy consumption is saved as much as possible without affecting system performance, a lower operating frequency can be set for each second processor, for example, the first operating frequency corresponding to the first processor is 1.5GHz, and the second operating frequency corresponding to the second processor is 1.0GHz. When an update of the intermediate data temporary storage area is detected, each second processor can read the intermediate features from the intermediate data temporary storage area, perform decoding processing, and generate a final decoding result. Furthermore, each second processor can poll to obtain the corresponding intermediate features and output the final decoding results in sequence according to the time sequence of the intermediate feature acquisition. For example, for a natural language processing model, the decoding process will be carried out word by word and step by step to generate the final text output.

[0105] This approach ensures that computationally intensive pre-population tasks run at a high frequency (the first frequency) to quickly complete calculations, while memory-intensive decoding tasks run at a lower frequency (the second frequency), reducing unnecessary energy consumption. This not only improves resource utilization efficiency but also reduces environmental impact, providing an effective solution for sustainable intelligent computing.

[0106] Step 105: output the decoding results of each second processor in sequence.

[0107] In some embodiments, in order to ensure that the final product of the model reasoning process, such as the translated sentence or the object label in the image, can be output accurately, the decoding result of each second processor can be output sequentially to effectively manage and control the concurrent access of multiple second processors to the intermediate features, ensure the sequentiality of the decoding results, and thus avoid sequence confusion.

[0108] In some embodiments, when storing the intermediate features in the intermediate data temporary storage area, a unique task number can be assigned to each sub-processing task, and the intermediate features can be stored in the intermediate data temporary storage area in the order of the task numbers. The second processor retrieves the intermediate features from the intermediate data temporary storage area in the order of the task numbers, decodes them, and outputs the decoding results in the order of the task numbers. This ensures that even if multiple second processors are operating simultaneously, they can still process the tasks in the predetermined order, thereby ensuring the sequentiality of the decoding results.

[0109] In some embodiments, a polling queue may be established to queue the intermediate features in the order in which the tasks are submitted. Each second processor sequentially retrieves the intermediate features from the polling queue for decoding, thereby ensuring the order of the decoding results.

[0110] In some embodiments, a mutex or other synchronization mechanism can be used to control access to the intermediate data temporary storage area, ensuring that only one second processor can obtain intermediate features from the intermediate data temporary storage area at a time. This prevents multiple second processors from accessing the intermediate data temporary storage area simultaneously, thereby preventing order confusion.

[0111] In some implementations, a timestamp or version number can be added to the intermediate features to indicate the order in which they are processed. The second processor then outputs the decoding results in the order of the timestamps or version numbers, ensuring that the decoding results are output in the correct order even in a concurrent environment.

[0112] The embodiment of the present application obtains tasks to be processed and determines an intelligent computing cluster for processing the tasks to be processed; according to a preset division rule, multiple first processors and multiple second processors are determined from multiple intelligent computing processors in the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency; pre-filling processing is performed on the tasks to be processed by the multiple first processors corresponding to the first operating frequency to obtain intermediate features, and the intermediate features are transmitted to an intermediate data temporary storage area shared by the first processor and the second processor; when an update of the intermediate data temporary storage area is detected, the intermediate features are obtained from the intermediate data temporary storage area by the multiple second processors corresponding to the second operating frequency, and the intermediate features are decoded to obtain a decoding result of each second processor; and the decoding result of each second processor is output in sequence. In this way, different intelligent computing processors can be divided for processing according to the sensitivity of the tasks to be processed to the operating frequency of the intelligent computing processor at different processing stages. The first processor executes the pre-filling task and the second processor executes the decoding task. The first operating frequency corresponding to the first processor is set to be greater than the second operating frequency corresponding to the second processor. In this way, the pre-filling task that is more sensitive to frequency can be processed by the first processor with a higher frequency to quickly complete data processing; the decoding task that is less sensitive to frequency can be processed by the second processor with a lower frequency, so as to reduce energy consumption as much as possible while maintaining high system performance, so as to effectively reduce the energy consumption of the processor without affecting performance, thereby saving energy at the root.

[0113] In some embodiments, it can be determined whether the task to be processed needs to be processed across cluster nodes based on the volume and bandwidth of the task to be processed. Under the condition that bandwidth allows, for large-scale distributed computing tasks (such as big data analysis, machine learning training, etc.), high-performance computing tasks (such as climate simulation, physical simulation, etc.), tasks with strong task dependencies, tasks that require load balancing, etc., while ensuring that cross-cluster node processing can bring performance improvements, the computing resources and storage resources of multiple cluster nodes can be used to improve processing efficiency. That is, the pre-filling stage tasks of the task to be processed are placed on the first cluster node for processing, and the decoding stage tasks of the same task to be processed are placed on the second cluster node for processing, and so on.

[0114] In some embodiments, to reduce the data transmission delay of the intermediate features from the pre-filling stage to the decoding stage, the pending tasks can be processed in the same cluster node to ensure fast data access and transmission and maintain the efficiency of the overall processing flow. For example, step 102 may include:

[0115] (102.1) Determine a cluster node from the intelligent computing cluster for processing the task to be processed;

[0116] (102.2) Determining a first division ratio and a second division ratio for the plurality of intelligent computing processors of the cluster nodes according to a preset division rule;

[0117] (102.3) Determine a plurality of first processors from the plurality of intelligent computing processors according to the first division ratio;

[0118] (102.4) According to the second division ratio, a plurality of second processors are determined from the plurality of intelligent computing processors, and the total number of the plurality of first processors and the plurality of second processors is equal to the total number of intelligent computing processors of the cluster node.

[0119] Among them, a cluster node can be a physical or logical unit in an intelligent computing cluster, which can include multiple intelligent computing processors. Each cluster node is responsible for performing specific computing tasks and can run independently or work in collaboration with other nodes.

[0120] The first division ratio may be a ratio allocated to the first processor when determining the allocation of intelligent computing processors within the cluster node.

[0121] The second division ratio may be a ratio allocated to the second processor when determining the allocation of intelligent computing processors within the cluster node.

[0122] For example, a cluster node in an idle state may be selected from the intelligent computing cluster to process the task to be processed, or a cluster node with a smaller load may be selected to process the task to be processed.

[0123] For example, the preset partitioning rule may correspond to a fixed partitioning ratio, for example, setting a first partitioning ratio corresponding to the first processor to 50%, setting a second partitioning ratio corresponding to the second processor to 50%, and so on. In this case, when there are 10 intelligent computing processors in the cluster node that are idle, 5 are allocated to the first processor and 5 are allocated to the second processor.

[0124] For example, an intelligent scheduling algorithm can be used to analyze the characteristics of pending tasks, categorize them, and establish a database containing the typical characteristics of each task type. For each task type, the corresponding number of computational operations and memory accesses are recorded. The first and second processors for the current pending task are then divided based on the allocation ratios of the first and second processors for each task type recorded in the database.

[0125] In some embodiments, the number of computing operations when the task to be processed is in the pre-filling stage can also be predicted, and the number of memory accesses when the task to be processed is in the decoding stage can be predicted. Afterwards, the ratio of the first processor to the second processor is calculated based on the number of computing operations and the number of memory accesses. For example, if the number of computing operations is 10,000 when the task to be processed is in the pre-filling stage, and the number of memory accesses is 5,000 when the task to be processed is in the decoding stage, then the ratio of the first processor can be 10,000 / (10,000+5,000)≈2 / 3; the ratio of the second processor can be 5,000 / (10,000+5,000)≈1 / 3. Then, when a cluster node has 8 intelligent computing processors, 5 of them can be set as the first processor and the other 3 as the second processor. Furthermore, the prediction of the number of computing operations when the task to be processed is in the pre-filling stage can be made by a machine learning algorithm (such as a regression model, a neural network, a decision tree, etc.), or it can be made based on historical data. The embodiments of the present application do not impose specific restrictions on this.

[0126] Exemplarily, when the cluster node is in an idle state, the total number of multiple first processors and multiple second processors is equal to the total number of intelligent computing processors of the cluster node; when the cluster node is in a non-idle state, the total number of multiple first processors and multiple second processors is equal to the total number of intelligent computing processors of the cluster node in the idle state.

[0127] In some embodiments, the task type (such as natural language processing, image processing, speech recognition, etc., which may affect computational intensity and memory access intensity), task scale (the input data size of the task, such as text length, image resolution, etc.), and task complexity (the number of required operations, such as the number of matrix multiplications, the number of convolution operations, etc.) of the task to be processed can be obtained, and clustering can be performed in historical data based on the task type, task scale, and task complexity. The Euclidean distance between the task type of the task to be processed and the historical task type, the task scale and the historical task scale, and the task complexity and the historical task complexity can be calculated, and summed according to their respective weights to find the number of divisions of the first processor and the second processor of the k historical tasks with the smallest Euclidean distance, and the number of divisions of the first processor of the k historical tasks can be averaged to determine a plurality of first processors for processing the task to be processed; and the number of divisions of the second processor of the k historical tasks can be averaged to determine a plurality of second processors for processing the task to be processed.

[0128] By deploying the first processor and the second processor on the same cluster node and dynamically determining their ratio and number according to preset division rules, it is possible to optimize resource allocation, reduce data transmission delay, and reduce energy consumption, while improving processing efficiency and the overall performance of the system, ensuring that the intelligent computing cluster is both efficient and energy-saving when processing various computing tasks.

[0129] In some implementations, in order to dynamically balance the workload of the first processor and the second processor, ensure efficiency matching in the pre-filling and decoding processing stages, and avoid resource waste or bottlenecks caused by uneven processing speeds, by comparing the average latency of different processors and adjusting the number of processors accordingly, the system can be ensured to adaptively respond to workload changes, thereby optimizing the efficiency and response time of the overall task processing process. For example, the task processing method based on the intelligent computing cluster may also include:

[0130] (A.1) in the process of processing the pending task, regularly obtaining the average delay of the first character of the pending task performed by the plurality of first processors in pre-filling the pending task, and the average delay of the second character of the pending task performed by the plurality of second processors in decoding the pending task;

[0131] (A.2) comparing the average delay of the first character with the average delay of the second character to obtain a first comparison result;

[0132] (A.3) When the first comparison result indicates that the average delay of the first character is inconsistent with the average delay of the second character, the number of the plurality of first processors and the number of the plurality of second processors are adjusted according to the first comparison result.

[0133] The average first-character delay may be the average time delay for processing each character by multiple first processors during the pre-filling process of the pending task. The average first-character delay can be calculated by counting the total time and total number of characters processed by all first processors within a certain time interval. The average first-character delay reflects the processing speed and efficiency of the pre-filling task.

[0134] The average second-character delay may be the average time delay for processing each character by multiple second processors during the decoding process of the task to be processed. The average second-character delay can also be calculated by counting the total time and total number of characters processed by all second processors within a certain time interval. The average second-character delay reflects the processing speed and efficiency of the decoding task.

[0135] The first comparison result may be a comparison of the average first-character delay and the average second-character delay. The purpose of this comparison is to assess whether the processing speeds of the pre-filling task and the decoding task are compatible. If there is a significant difference between the average first-character delay and the average second-character delay, this indicates inconsistent processing speeds for the two tasks, and the number of processors may need to be adjusted to optimize overall performance.

[0136] Exemplarily, the first character delay may be determined by the ratio of the sum of the processing time of each first processor processing the pre-filling task to the total number of characters processed by all pre-filling tasks within a fixed time period.

[0137] Specifically, the calculation formula for the first character delay is as follows:

[0138]

[0139] Among them, T1 i is the first character delay of the i-th first processor processing the pre-fill task, n is the total number of first processors, and c is the total number of characters processed by all pre-fill tasks.

[0140] The calculation formula for the second character delay is as follows:

[0141]

[0142] Among them, T2 i is the second character delay of the i-th second processor processing the pre-fill task, m is the total number of first processors, and d is the total number of characters processed by all decoding tasks.

[0143] For example, if the average delay for the first character is 10 milliseconds and the average delay for the second character is 5 milliseconds, comparing the average delay for the first character and the average delay for the second character to obtain a first comparison result can indicate that the average delay for the first character is inconsistent with the average delay for the second character. The average delay for multiple first processors to handle pre-filling tasks is much greater than the average delay for multiple first processors to handle decoding tasks, indicating that the workload of the first processors is heavy. To balance the load, the number of first processors can be increased, for example, from 5 to 6, to reduce the average delay of each first processor, while the number of second processors can be reduced, for example, from 5 to 4, because the current second processors are less loaded.

[0144] In some embodiments, a prediction model can be introduced to perform extensive training based on historical pending tasks to predict the delay trend of pending tasks, and the number of first processors and second processors can be adjusted in advance to cope with possible load changes and reduce the delay caused by real-time adjustments.

[0145] Through the above methods, resource allocation can be dynamically adjusted according to the actual processing situation to ensure the efficient operation of the intelligent computing cluster, reduce unnecessary waiting time, and improve the overall processing speed and resource utilization.

[0146] In some implementations, to dynamically balance the workloads of the first and second processors and ensure the efficiency and responsiveness of the entire task processing flow, the average first character delay and the average second character delay can be compared and the number of processors adjusted based on the difference. This can effectively reduce processing bottlenecks, avoid resource waste, and improve overall system throughput and performance. For example, "adjusting the number of the plurality of first processors and the plurality of second processors based on the first comparison result" in (A.3) may include:

[0147] (A.3.1) When the first comparison result indicates that the average delay of the first character is greater than the average delay of the second character, obtaining a first difference between the average delay of the first character and the average delay of the second character;

[0148] (A.3.2) Determining a first number of first processors to be added based on a first difference range corresponding to the first difference, and adjusting the number of the plurality of first processors and the plurality of second processors based on the first number;

[0149] (A.3.3) When the first comparison result indicates that the average delay of the first character is less than the average delay of the second character, obtaining a second difference between the average delay of the second character and the average delay of the first character;

[0150] (A.3.4) Determine a second number of additional second processors based on a second difference range corresponding to the second difference, and adjust the number of the plurality of first processors and the plurality of second processors based on the second number.

[0151] The first difference may be the difference between the first character average delay and the second character average delay when the first character average delay is greater than the second character average delay. The calculation formula is: first difference = first character average delay - second character average delay.

[0152] The first difference range can be a series of predefined threshold intervals used to classify the first difference to determine the number of first processors to be added. Each first difference range corresponds to a specific first quantity. For example, first difference range 1: 50 < first difference ≤ 100ms; first difference range 2: 100ms < first difference ≤ 300ms; first difference range 3: 300ms < first difference ≤ 150ms.

[0153] The first number may be the number of first processors that need to be added determined according to a range of the first difference.

[0154] The second difference may be the difference between the second character average delay and the first character average delay when the first character average delay is smaller than the second character average delay. The calculation formula is: second difference = second character average delay - first character average delay.

[0155] The second difference range can be a series of predefined threshold intervals used to classify the second difference to determine the number of second processors to be added. Each second difference range corresponds to two specific second quantities. For example, second difference range 1: 50 < second difference ≤ 100ms; second difference range 2: 100ms < second difference ≤ 300ms; and second difference range 3: 300ms < second difference ≤ 150ms.

[0156] The second number may be the number of second processors that need to be added determined according to a range of the second difference.

[0157] For example, if the first difference is within the range of 50 < first difference ≤ 100ms, one first processor is added. If the first difference is within the range of 100ms < first difference ≤ 300ms, two first processors are added. If the first difference is within the range of 300ms < first difference, three first processors are added. It is understandable that

[0158] For example, if the second difference is within the range of 100 < second difference ≤ 200 ms, one second processor is added. If the second difference is within the range of 200 ms < second difference ≤ 300 ms, two second processors are added. If the second difference is within the range of 300 ms < second difference, three second processors are added.

[0159] In some embodiments, the first difference range, the first quantity, the second difference range, and the second quantity can all be set according to actual conditions. For example, the first difference range can also be in the range of 5<first difference≤10ms, 10<first difference≤20ms, 20<first difference≤50ms, etc. This application does not make any specific limitations on this.

[0160] For example, since the number of intelligent computing processors in the cluster node is fixed, when the number of first processors is increased, the number of second processors will be reduced by the same amount. For example, the original number of first processors is 5 and the number of second processors is 5. After adding 1 first processor, the number of second processors will be reduced by 1; similarly, when the number of second processors is increased, the number of first processors will be reduced by the same amount.

[0161] Dynamically adjusting the number of the first processor and the second processor can help achieve optimal resource allocation, improve system processing capabilities, ensure efficient and stable task processing, and reduce operating costs.

[0162] In some implementations, in order to flexibly adjust resource usage based on actual load and performance requirements while ensuring performance, the operating frequency of the second processor can be dynamically adjusted based on the actual system performance and preset goals to achieve an optimal balance between performance and energy consumption. For example, after step (A.3), the following steps may also be included:

[0163] (B.1) Obtain the sum of the average delay of the first character and the average delay of the second character to obtain the total delay;

[0164] (B.2) obtaining a preset delay threshold, comparing the delay threshold with the total delay, and obtaining a second comparison result;

[0165] (B.3) When the second comparison result indicates that the total delay is greater than the delay threshold, determining a third operating frequency corresponding to the second processor, and obtaining intermediate features from the intermediate data temporary storage area using multiple second processors corresponding to the third operating frequency, and decoding the intermediate features to obtain a decoding result for each second processor; wherein the third operating frequency is greater than the second operating frequency;

[0166] (B.4) When the second comparison result indicates that the total delay is less than the delay threshold, determine the fourth operating frequency corresponding to the second processor, and obtain intermediate features from the intermediate data temporary storage area through multiple second processors corresponding to the fourth operating frequency, and decode the intermediate features to obtain a decoding result of each second processor; wherein the fourth operating frequency is less than the second operating frequency.

[0167] The total delay may be the sum of the average delay of the first character and the average delay of the second character, and is calculated as follows: total delay = average delay of the first character + average delay of the second character.

[0168] The delay threshold may be a preset upper limit of delay, which is used to evaluate whether the total delay is within an acceptable range. The delay threshold may be set according to actual application scenarios and performance requirements.

[0169] The second comparison result may be a comparison of the total delay with a delay threshold, used to determine whether the total delay exceeds an allowable range. If the total delay is greater than the delay threshold, it indicates that the current processing speed is slow and the response time needs to be optimized. If the total delay is less than the delay threshold, it indicates that the current processing speed is fast and energy efficiency can be appropriately reduced to conserve resources.

[0170] The third operating frequency may be a higher operating frequency set for the second processor when the total delay is greater than the delay threshold. By increasing the operating frequency, the processing speed of the decoding task may be accelerated to a certain extent, thereby reducing the total delay.

[0171] The fourth operating frequency may be a lower operating frequency set for the second processor when the total delay is less than the delay threshold. By lowering the operating frequency, energy can be saved and energy efficiency can be improved.

[0172] In some embodiments, after adjusting the number of first and second processors, the average latency difference between pre-filling tasks and decoding tasks is within an allowable range. In this case, when the total latency exceeds a latency threshold, it indicates that the current processing speed of the pending tasks is insufficient. Since the operating frequency of the second processor can speed up decoding to a certain extent, the operating frequency of the second processor can be increased to speed up decoding to a certain extent and reduce latency. When the total latency is below the threshold, it indicates that the processing speed is relatively fast. The operating frequency of the second processor can be appropriately reduced to reduce energy consumption, thereby achieving a balance between performance and energy consumption.

[0173] For example, when the sum of the average delay of the first character and the average delay of the second character is 70 milliseconds and the delay threshold is 50 milliseconds, the total delay is greater than the delay threshold. At this time, the operating frequency of each second processor can be increased. For example, if the initial second operating frequency is 1.0 GHz, the third operating frequency can be adjusted to 1.2 GHz.

[0174] For example, when the sum of the average delay of the first character and the average delay of the second character is 30 milliseconds and the delay threshold is 50 milliseconds, the total delay is less than the delay threshold. At this time, the operating frequency of each second processor can be reduced. For example, if the initial second operating frequency is 1.0 GHz, the fourth operating frequency can be adjusted to 0.8 GHz.

[0175] In some embodiments, the amplitude of adjusting the operating frequency of the second processor can be a set granularity, such as adjusting 10% each time, or it can be a dynamic calculation result based on a prediction model. The embodiments of the present application do not impose specific restrictions on this.

[0176] When processing speed is insufficient, the second processor's frequency is increased to speed up decoding, reduce latency, and ensure timely task processing. When processing speed is high, the frequency is reduced to reduce energy consumption and achieve energy efficiency. This not only improves the system's adaptability to different loads, but also achieves an intelligent balance between performance and energy consumption, thereby improving overall system efficiency and responsiveness.

[0177] In some embodiments, in order to optimize the energy efficiency ratio of the processors in the intelligent computing cluster, the maximum operating frequency of the first processor can be set, and the total power consumption and total delay of the system of the second processor at different operating frequencies can be predicted. Based on this, the power consumption and performance curves can be drawn to determine the optimal operating frequency of the second processor. This can achieve the optimal balance between performance and energy consumption while ensuring processing performance. For example, the second operating frequency can be determined by:

[0178] (C.1) setting the first operating frequency corresponding to the plurality of first processors as the highest operating frequency;

[0179] (C.2) predicting, based on the maximum operating frequency, the total operating power consumption and total operating latency of the cluster nodes corresponding to the plurality of first processors and the plurality of second processors at different operating frequencies of the plurality of second processors;

[0180] (C.3) Draw a power consumption curve based on the total power consumption of the operation, and draw a performance curve based on the total latency of the operation;

[0181] (C.4) Determine second operating frequencies of the plurality of second processors based on the power consumption curve and the performance curve.

[0182] The maximum operating frequency may be the maximum operating frequency used by the first processor when executing the pre-filling task, and the maximum operating frequency is set in order to maximize computing performance.

[0183] Among them, the total operating power consumption can be the total electrical energy consumed by all intelligent computing processors in the cluster node when executing tasks at different operating frequencies of the second processor. The total operating power consumption can be obtained through prediction and can be expressed in watts (W).

[0184] The total operating delay may be the time required for the cluster node to process the pending task until the task is completely completed under different operating frequencies of the second processor, that is, the processing time including the pre-filling task and the decoding task, and may be expressed in milliseconds (ms).

[0185] Among them, the power consumption curve can be a trend chart of the total operating power consumption of the cluster node changing with the operating frequency at different operating frequencies of the second processor. By drawing the power consumption curve, the power consumption at different operating frequencies of the second processor can be intuitively seen, which helps to select the optimal second operating frequency.

[0186] The performance curve can be a trend chart showing the total latency of the cluster nodes as a function of the second processor's operating frequency. By plotting the performance curve, one can intuitively see the performance of the second processor at different operating frequencies, helping to select the optimal second operating frequency.

[0187] For example, first, the highest operating frequency of the first processor (responsible for the pre-filling process) can be determined as the first operating frequency. For example, if the highest frequency at which the first processor can stably operate is 2.5 GHz, then 2.5 GHz can be used as the first operating frequency of the first processor to ensure that the pre-filling process can be completed at the fastest speed.

[0188] In some embodiments, a neural network model can be used to predict the total operating power consumption and total operating latency of a first processor operating at a first operating frequency and a second processor operating at different operating frequencies based on historical data. For example, the historical total operating power consumption and historical total operating latency of a historical task at different second operating frequencies under the first operating frequency can be determined. Each initial second operating frequency can correspond to multiple historical tasks, and from the multiple historical tasks, a target historical task with the highest similarity to the task to be processed is selected.

[0189] Furthermore, when selecting the target historical task with the highest similarity to the task to be processed, the number of first processors and second processors corresponding to the task to be processed, the task type of the task to be processed (for example, natural language processing, image processing, speech recognition, etc., the task type can affect the computational intensity and memory access intensity), the task scale (the input data size of the task, for example, text length, image resolution, etc.), and the task complexity (the number of operations required, such as the number of matrix multiplications, the number of convolution operations, etc.) can be obtained. Based on the number of first processors and second processors, the task type, the task scale, and the task complexity, the target historical task with the highest similarity to the task to be processed is selected from the historical tasks corresponding to each initial second operating frequency, and the historical total operating power consumption and historical total operating delay corresponding to the target historical task are used as the total operating power consumption and total operating delay of the cluster node when the second processor operates at the second operating frequency.

[0190] Furthermore, the power consumption and latency of the second processor (responsible for decoding) at different operating frequencies can be predicted. For example, the total power consumption and latency of the intelligent computing cluster at the second operating frequencies of 1.0 GHz, 1.5 GHz, and 2.0 GHz can be predicted, and power consumption and performance curves can be drawn based on the predicted results.

[0191] Furthermore, by analyzing the power consumption curve and the performance curve, if the power consumption and performance of the second processor reach a good balance point when the second operating frequency of the second processor is 1.5 GHz, the power consumption does not increase much compared to the lowest frequency, while the performance (processing speed) is significantly improved, then 1.5 GHz can be used as the second operating frequency.

[0192] By setting the maximum operating frequencies of multiple first processors and predicting the total power consumption and latency at different frequencies, a power consumption and performance curve can be drawn, thereby optimizing the selection of the best operating frequency of the second processor and achieving the optimal balance between power consumption and performance.

[0193] In some implementations, to ensure orderly processing of tasks, intermediate features can be stored in an intermediate data temporary storage area according to the task number to ensure the orderliness and consistency of the data. For example, the task processing method based on the intelligent computing cluster can also include:

[0194] (D.1) Divide the task to be processed into multiple sub-tasks and number the multiple sub-tasks according to the task sequence;

[0195] (D.2) In the intermediate data temporary storage area, the received intermediate features are stored according to the task number of each sub-processing task, and the intermediate features are sequentially obtained from the intermediate data temporary storage area according to the corresponding task numbers through multiple second processors.

[0196] Subtasks can be subdivided into tasks to increase parallelism. This means splitting a task into multiple smaller, independent processing units. Each subtask can be pre-populated and decoded independently. This division facilitates parallel processing and resource optimization, improving overall task processing efficiency.

[0197] The task number can be a unique identifier assigned to each sub-task to distinguish and manage different sub-tasks. The task number is typically an increasing integer, numbered sequentially according to the order in which the tasks are processed, to maintain order and consistency during the processing process.

[0198] For example, if the task to be processed is a natural language processing task, the input data contains 100 sentences, each sentence contains 100 characters, 5 first processors are used for pre-filling processing, and 3 second processors are used for decoding processing. In this case, 100 sub-processing tasks can be divided, with each sentence as a sub-processing task.

[0199] Furthermore, after the first processor completes processing the corresponding sentence and obtains the intermediate feature, it can use the sub-processing task number to correspond. For example, if the intermediate feature corresponds to sub-processing task 1, then its corresponding task number is also 1, thereby obtaining intermediate feature 1, intermediate feature 2,..., intermediate feature 100.

[0200] Furthermore, 100 sub-processing tasks can be assigned to 3 second processors, and the allocation order can be determined according to the order of the serial numbers. For example, intermediate feature 1 is assigned to second processor 1, intermediate feature 2 is assigned to second processor 2, intermediate feature 3 is assigned to second processor 3, and then the cycle starts from second processor 1, intermediate feature 4 is assigned to second processor 1, and so on.

[0201] Furthermore, when a second processor completes decoding and obtains a decoding result, the decoding result of each second processor can be output in sequence, for example, the decoding result of intermediate feature 1 is output first, then the decoding result of intermediate feature 2, and so on.

[0202] In some embodiments, queue-based task scheduling, the use of tags and flags, and task scheduling management through dependency graphs can also be used to achieve reasonable task dispatching. For example, pipeline technology can be used to utilize processors in multiple stages to keep tasks flowing in a fixed order at different stages. After each stage is processed, the task will be passed to the next stage in a predetermined order. The embodiments of the present application do not impose too many restrictions on this.

[0203] Through the above methods, the order and consistency of task processing in cluster nodes can be ensured, and the effectiveness and predictability of data management can be improved.

[0204] See also Figure 3 The embodiment of the present application further provides a task processing device based on an intelligent computing cluster, which can implement the above-mentioned task processing method based on an intelligent computing cluster. The task processing device based on an intelligent computing cluster includes:

[0205] An acquisition module 31 is used to acquire tasks to be processed and determine an intelligent computing cluster for processing the tasks to be processed;

[0206] a determination module 32, configured to determine, according to a preset partitioning rule, a plurality of first processors and a plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency;

[0207] A processing module 33 is configured to perform pre-fill processing on the task to be processed by using the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmit the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor;

[0208] The decoding module 34 is configured to, when detecting that the intermediate data temporary storage area is updated, obtain intermediate features from the intermediate data temporary storage area through the plurality of second processors corresponding to the second operating frequency, and decode the intermediate features to obtain decoding results of each second processor;

[0209] The output module 35 is configured to sequentially output the decoding results of each second processor.

[0210] The specific implementation of the task processing device based on the intelligent computing cluster is basically the same as the specific implementation of the task processing method based on the intelligent computing cluster, and will not be repeated here. On the premise of meeting the requirements of the embodiment of this application, the task processing device based on the intelligent computing cluster can also be provided with other functional modules to implement the task processing method based on the intelligent computing cluster in the above embodiment.

[0211] The present application also provides a computer device comprising a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned task processing method based on an intelligent computing cluster. The computer device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0212] See also Figure 4 , Figure 4 The hardware structure of a computer device according to another embodiment is shown. The computer device includes:

[0213] The processor 41 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0214] The memory 42 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 42 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program codes are stored in the memory 42 and are called by the processor 41 to execute the task processing method based on the intelligent computing cluster in the embodiments of this application.

[0215] Input / output interface 43, used for information input and output;

[0216] Communication interface 44, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0217] bus 45 , which transmits information between the various components of the device (e.g., processor 41 , memory 42 , input / output interface 43 , and communication interface 44 );

[0218] The processor 41 , the memory 42 , the input / output interface 43 and the communication interface 44 are connected to each other in communication within the device via a bus 45 .

[0219] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned task processing method based on the intelligent computing cluster.

[0220] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0221] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0222] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0223] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0224] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0225] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0226] It should be understood that in this application, "at least one (item)" and "several" refer to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0228] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0230] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0231] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A task processing method based on intelligent computing cluster, characterized in that: The method comprises: Obtaining pending tasks and determining an intelligent computing cluster for processing the pending tasks; Determine, according to a preset partitioning rule, a plurality of first processors and a plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency; Pre-filling the to-be-processed task by the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmitting the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor; When an update of the intermediate data temporary storage area is detected, the intermediate features are obtained from the intermediate data temporary storage area by the multiple second processors corresponding to the second operating frequency, and the intermediate features are decoded to obtain a decoding result of each second processor; outputting the decoding results of each second processor in sequence; The method also includes: in the process of processing the tasks to be processed, regularly obtaining the average first character delay of the multiple first processors performing pre-fill processing on the tasks to be processed, and the average second character delay of the multiple second processors performing decoding processing on the tasks to be processed; comparing the first character average delay with the second character average delay to obtain a first comparison result; when the first comparison result indicates that the first character average delay and the second character average delay are inconsistent, adjusting the number of the multiple first processors and the multiple second processors according to the first comparison result.

2. The task processing method based on intelligent computing cluster according to claim 1, characterized in that: The first processor and the second processor are located in the same cluster node; and determining the plurality of first processors and the plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster according to a preset partitioning rule includes: Determining a cluster node for processing the task to be processed from the intelligent computing cluster; Determining a first division ratio and a second division ratio for the plurality of intelligent computing processors of the cluster node according to a preset division rule; Determining a plurality of first processors from the plurality of intelligent computing processors according to the first division ratio; According to the second division ratio, a plurality of second processors are determined from the plurality of intelligent computing processors, and the total number of the plurality of first processors and the plurality of second processors is equal to the total number of intelligent computing processors of the cluster node.

3. The task processing method based on intelligent computing cluster according to claim 1, characterized in that: The adjusting the number of the plurality of first processors and the plurality of second processors according to the first comparison result includes: When the first comparison result indicates that the average delay of the first character is greater than the average delay of the second character, obtaining a first difference between the average delay of the first character and the average delay of the second character; determining a first number of first processors to be added according to a first difference range corresponding to the first difference, and adjusting the number of the plurality of first processors and the plurality of second processors according to the first number; When the first comparison result indicates that the average delay of the first character is less than the average delay of the second character, obtaining a second difference between the average delay of the second character and the average delay of the first character; A second number of added second processors is determined according to a second difference range corresponding to the second difference, and the number of the plurality of first processors and the plurality of second processors is adjusted according to the second number.

4. The task processing method based on intelligent computing cluster according to claim 1, characterized in that: When the first comparison result indicates that the first character average delay and the second character average delay are inconsistent, after adjusting the number of the plurality of first processors and the plurality of second processors according to the first comparison result, the method further includes: Obtaining the sum of the average delay of the first character and the average delay of the second character to obtain a total delay; Obtaining a preset delay threshold, and comparing the delay threshold with the total delay to obtain a second comparison result; When the second comparison result indicates that the total delay is greater than the delay threshold, determining a third operating frequency corresponding to the second processor, and obtaining the intermediate features from the intermediate data temporary storage area by the multiple second processors corresponding to the third operating frequency, and decoding the intermediate features to obtain a decoding result of each second processor; wherein the third operating frequency is greater than the second operating frequency; When the second comparison result indicates that the total delay is less than the delay threshold, the fourth operating frequency corresponding to the second processor is determined, and the intermediate features are obtained from the intermediate data temporary area through the multiple second processors corresponding to the fourth operating frequency, and the intermediate features are decoded to obtain the decoding results of each second processor; wherein, the fourth operating frequency is less than the second operating frequency.

5. The task processing method based on intelligent computing cluster according to claim 1, characterized in that: The second operating frequency is determined by: Setting the first operating frequencies corresponding to the plurality of first processors as the highest operating frequency; Based on the highest operating frequency, predicting the total operating power consumption and total operating delay of the cluster nodes corresponding to the multiple first processors and the multiple second processors at different operating frequencies of the multiple second processors; Drawing a power consumption curve based on the total power consumption of the operation, and drawing a performance curve based on the total delay of the operation; Based on the power consumption curve and the performance curve, second operating frequencies of the plurality of second processors are determined.

6. The task processing method based on intelligent computing cluster according to claim 1, characterized in that: The method further comprises: Divide the task to be processed into multiple sub-tasks, and number the multiple sub-tasks according to the order of tasks; In the intermediate data temporary storage area, the received intermediate features are stored according to the task number of each sub-processing task, and the intermediate features are sequentially obtained from the intermediate data temporary storage area according to the corresponding task numbers through the multiple second processors.

7. A task processing device based on an intelligent computing cluster, characterized in that: The device comprises: An acquisition module is used to acquire tasks to be processed and determine an intelligent computing cluster for processing the tasks to be processed; a determination module, configured to determine, according to a preset partitioning rule, a plurality of first processors and a plurality of second processors from the plurality of intelligent computing processors of the intelligent computing cluster, wherein each first processor corresponds to a first operating frequency, each second processor corresponds to a second operating frequency, and the first operating frequency is greater than the second operating frequency; a processing module, configured to perform pre-filling processing on the to-be-processed task by using the plurality of first processors corresponding to the first operating frequency to obtain intermediate features, and transmit the intermediate features to an intermediate data temporary storage area shared by the first processor and the second processor; a decoding module configured to, when detecting that the intermediate data temporary storage area is updated, obtain the intermediate features from the intermediate data temporary storage area through the plurality of second processors corresponding to the second operating frequency, and decode the intermediate features to obtain a decoding result of each second processor; an output module, configured to sequentially output the decoding results of each second processor; The device is also used to: in the process of processing the tasks to be processed, regularly obtain the average first character delay of the multiple first processors performing pre-filling processing on the tasks to be processed, and the average second character delay of the multiple second processors performing decoding processing on the tasks to be processed; compare the first character average delay with the second character average delay to obtain a first comparison result; when the first comparison result indicates that the first character average delay and the second character average delay are inconsistent, adjust the number of the multiple first processors and the multiple second processors according to the first comparison result.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the task processing method based on the intelligent computing cluster according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the task processing method based on an intelligent computing cluster according to any one of claims 1 to 6 is implemented.

Images