Calculation power scheduling method and device for model training, storage medium and electronic equipment

Abstract

The invention provides a computing power scheduling method and device for model training, a computer storage medium and electronic equipment, and relates to the technical field of computers. Comprising the following steps: in response to a received training task request for a to-be-trained model, dividing the to-be-trained model into a plurality of first model subtasks; aiming at the target model subtask, determining a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from a heterogeneous computing power architecture according to a pre-configured computing power mapping table; the computing power mapping table comprises a one-to-one correspondence relationship between each computing task and the heterogeneous computing power unit of the corresponding resource type; and determining a target heterogeneous computing power unit from heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and allocating the target model subtask to the target heterogeneous computing power unit, so as to execute the target model subtask based on the target heterogeneous computing power unit. The method can improve the model training efficiency and the accuracy of the model prediction result.

Description

Background Art

[0002] With the rapid development of computer technology and artificial intelligence, big data computing and big model training have played an irreplaceable and important role in the application fields of various industries. For example, in the field of modern heating, the accuracy of heating load forecasting is directly related to the energy consumption, operating costs and user comfort of the heating system. The current large thermal model training has greatly improved the accuracy of heating load forecasting.

[0003] However, due to the large number of parameters and extremely high computational complexity involved in the model training process, the traditional single computing power architecture has been unable to meet the needs of large-scale thermal big data calculations and thermal large model training, resulting in problems such as long model training time and low prediction result accuracy. Summary of the invention

[0004] The present invention provides a computing power scheduling method for model training, a computing power scheduling device for model training, a computer storage medium and an electronic device, thereby reducing the time consumption of model training and improving the efficiency of model training and the accuracy of model prediction results.

[0005] In a first aspect, an embodiment of the present disclosure provides a computing power scheduling method for model training, the method comprising: in response to receiving a training task request for a model to be trained, dividing the model to be trained into multiple first model subtasks; for each target model subtask in the multiple first model subtasks, determining, according to a pre-configured computing power mapping table, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from a heterogeneous computing power architecture; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table contains a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; determining a target heterogeneous computing power unit from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and allocating the target model subtask to the target heterogeneous computing power unit, so as to execute the target model subtask based on the target heterogeneous computing power unit.

[0006] In an optional embodiment of the present disclosure, dividing the model to be trained into a plurality of first model subtasks includes: dividing the model to be trained according to preset parameters to obtain a plurality of first model subtasks; wherein the preset parameters include at least one or more of the following: Real-time demand parameters; Task magnitude parameters; Data type parameter; Parameters of idle computing resources of each heterogeneous computing unit.

[0007] In an optional embodiment of the present disclosure, the method further includes: determining, for each target computing task in each computing task, a computing speed at which heterogeneous computing power units of each resource type execute the target computing task; determining the heterogeneous computing power unit of the resource type with the fastest computing speed as the heterogeneous computing power unit of the resource type that matches the target computing task, so as to construct a computing power mapping table.

[0008] In an optional embodiment of the present disclosure, multiple first model subtasks include at least a model building task, and the method also includes: obtaining a target training data set; determining data features of each training data contained in the target training data set; constructing a model structure of the model to be trained according to the data features to execute the model building task, and determining a heterogeneous computing power unit of a target resource type corresponding to the model building task from a heterogeneous computing power architecture; wherein the model structure of the model to be trained includes at least one or more of the following: a trend analysis model structure, a regression analysis model structure, a time series model structure, a machine learning model structure, a mechanism and data fusion model structure, and a grey system model structure.

[0009] In an optional embodiment of the present disclosure, obtaining a target training data set includes: performing a preprocessing operation on each training data in an initial training data set to obtain a preprocessed target training data set; wherein the preprocessing operation includes at least one or more of the following: the preprocessing operation includes at least one of the following: data cleaning processing, data denoising processing, normalization processing, data standardization processing, data enhancement processing, and missing value filling processing.

[0010] In an optional embodiment of the present disclosure, a target heterogeneous computing power unit is determined from heterogeneous computing power units of the target resource type of each computing node in a heterogeneous computing power architecture, and a target model subtask is allocated to the target heterogeneous computing power unit, including: if the target heterogeneous computing power unit is in multiple computing nodes in the heterogeneous computing power architecture, the target model subtask is divided into multiple second model subtasks to allocate the multiple second model subtasks to the target heterogeneous computing power units in the multiple computing nodes.

[0011] In an optional embodiment of the present disclosure, multiple first model subtasks include at least a third model subtask, and the method further includes: executing a fourth model subtask among the multiple first model subtasks based on the heterogeneous computing power units of each computing node in the heterogeneous computing power architecture to generate a target training model; the fourth model subtask is the first model subtask executed before the third model subtask; determining a model prediction result according to the target training model; in response to a loss value between the model prediction result and the actual result being greater than a preset loss value, executing the third model subtask to update the target training model based on the third model subtask until the loss value between the model prediction result and the actual result of the updated target training model is no greater than the preset loss value.

[0012] In a second aspect, an embodiment of the present disclosure provides a computing power scheduling device for model training, the device comprising: a task division module, for dividing the model to be trained into multiple first model subtasks in response to receiving a training task request for the model to be trained; a computing power determination module, for determining, for each target model subtask in the multiple first model subtasks, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from a heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table includes a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; a task allocation module, for determining a target heterogeneous computing power unit from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and allocating the target model subtask to the target heterogeneous computing power unit, so as to execute the target model subtask based on the target heterogeneous computing power unit.

[0013] In an optional embodiment of the present disclosure, the task division module is specifically used to divide the model to be trained according to preset parameters to obtain multiple first model subtasks; wherein the preset parameters include at least one or more of the following: Real-time demand parameters; Task magnitude parameters; Data type parameter; Parameters of idle computing resources of each heterogeneous computing unit.

[0014] In an optional embodiment of the present disclosure, the device may also include a computing speed determination module and a mapping table construction module, wherein the computing speed determination module is used to determine the computing speed of heterogeneous computing power units of each resource type to execute the target computing task for each target computing task in each computing task; the mapping table construction module is used to determine the heterogeneous computing power unit of the resource type with the fastest computing speed as the heterogeneous computing power unit of the resource type matching the target computing task, so as to construct a computing power mapping table.

[0015] In an optional embodiment of the present disclosure, multiple first model subtasks include at least a model building task, and the device may also include a data acquisition module, a data feature determination module and a task execution module; the data acquisition module is used to acquire a target training data set; the data feature determination module is used to determine the data features of each training data contained in the target training data set; the task execution module is used to construct a model structure of the model to be trained according to the data features to execute the model building task, and determine the heterogeneous computing power unit of the target resource type corresponding to the model building task from the heterogeneous computing power architecture; wherein the model structure of the model to be trained includes at least one or more of the following: trend analysis model structure, regression analysis model structure, time series model structure, machine learning model structure, mechanism and data fusion model structure, and grey system model structure.

[0016] In an optional embodiment of the present disclosure, the data acquisition module is used to perform preprocessing operations on each training data in the initial training data set to obtain a preprocessed target training data set; wherein the preprocessing operation includes at least one of the following: data cleaning processing, data denoising processing, normalization processing, data standardization processing, data enhancement processing, and missing value filling processing.

[0017] In an optional embodiment of the present disclosure, a task allocation module is used to divide the target model subtask into multiple second model subtasks if the target heterogeneous computing power unit is in multiple computing nodes in the heterogeneous computing power architecture, so as to allocate the multiple second model subtasks to the target heterogeneous computing power unit in the multiple computing nodes.

[0018] In an optional embodiment of the present disclosure, a plurality of first model subtasks include at least a third model subtask, and the device further includes a model generation module and a model update module, wherein the model generation module is used to execute a fourth model subtask among the plurality of first model subtasks according to the heterogeneous computing power units of each computing node in the heterogeneous computing power architecture to generate a target training model; the fourth model subtask is the first model subtask executed before the third model subtask; a model prediction result is determined according to the target training model; and the model update module is used to execute the third model subtask in response to a loss value between the model prediction result and the actual result being greater than a preset loss value, so as to update the target training model based on the third model subtask until the loss value between the model prediction result and the actual result of the updated target training model is no greater than the preset loss value.

[0019] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the computing power scheduling method for model training as described above.

[0020] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the computing power scheduling method for model training as above by executing the executable instructions.

[0021] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which is executed by a processor to implement the computing power scheduling method for model training as described above.

[0022] The technical solution disclosed in this disclosure has the following beneficial effects: The computing power scheduling method for the above-mentioned model training divides the model to be trained into multiple first model subtasks in response to receiving a training task request for the model to be trained; for each target model subtask in the multiple first model subtasks, according to a pre-configured computing power mapping table, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask is determined from the heterogeneous computing power architecture; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table contains a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; the target heterogeneous computing power unit is determined from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and the target model subtask is allocated to the target heterogeneous computing power unit, so as to execute the target model subtask based on the target heterogeneous computing power unit.

[0023] On the one hand, this method splits the model training task for the model to be trained into multiple first model subtasks, so as to perform parallel task processing on multiple first model subtasks. Compared with the method of directly processing the model training task of the model to be trained in the related technical solution, it greatly reduces the time consumption in the model training process, thereby improving the efficiency of model training. On the other hand, for each first model subtask, this method allocates each first model subtask to a heterogeneous computing power unit of the corresponding resource type through a pre-configured computing power mapping table. Since heterogeneous computing power units of different resource types have different computing capabilities for different computing tasks, for example, a graphics processing unit (GPU) is better at processing image data (for example, processing image data faster and more accurately) than a heterogeneous computing power unit of the central processing unit (CPU) resource type. Therefore, the above method can make full use of the hardware resources of each heterogeneous computing unit, and allocate the best performance heterogeneous computing unit for processing according to the specific model task requirements, thereby optimizing the hardware resource utilization in the system, improving the flexibility of model training task processing, and greatly reducing the time consumption of model training. Improve the efficiency of model training, so as to better adapt to application fields with higher requirements for model training time.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 An application scenario architecture diagram of a computing power scheduling system for model training in one of the exemplary embodiments is schematically shown.

[0027] Figure 2 A flowchart of a computing power scheduling method for model training in one of the exemplary embodiments is schematically shown.

[0028] Figure 3 A flowchart of a method for constructing a computing power mapping table in this exemplary embodiment is schematically shown.

[0029] Figure 4A flowchart of another method for constructing a computing power mapping table in this exemplary embodiment is schematically shown.

[0030] Figure 5 A schematic diagram of the structure of a computing power scheduling device for model training in one of the exemplary embodiments of the present invention is shown schematically.

[0031] Figure 6 The structure diagram of one electronic device in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the exemplary embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0034] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0035] In order to help those skilled in the art better understand the technical solution of the present disclosure, the relevant contents involved in the technical solution of the present disclosure are introduced below.

[0036] 1) Heterogeneous computing architecture / heterogeneous computing: refers to a computing system composed of heterogeneous computing units of various resource types (such as central processing unit (CPU), graphics processing unit (GPU), field programmable gate array (FPGA), digital signal processor (DSP), application specific integrated circuit (ASIC), etc.). The system can flexibly use different types of computing resources according to different computing needs, thereby improving overall computing efficiency and performance.

[0037] 2) Homogeneous computing architecture / homogeneous computing power: refers to a computing method that uses computing nodes of the same type of instruction set and architecture to form a system. In this architecture, all computing nodes follow the same type of instruction set and architecture, and usually use a single processor type, such as CPU, GPU, etc.

[0038] In the relevant technical background, with the rapid development of computer technology and artificial intelligence, big data computing and big model training have played an irreplaceable and important role in the application fields of various industries. Taking the thermal field as an example, due to the acceleration of urbanization, heating load prediction is becoming more and more important in modern heating systems. The accuracy of heating load prediction is directly related to the energy consumption, operating cost and user comfort of the heating system. Traditional heating load prediction methods mostly rely on statistical regression models, which often have great limitations when dealing with complex nonlinear problems. In recent years, deep learning, as an effective nonlinear modeling tool, has gradually been applied to heating load prediction. For example, deep neural networks (DNN), convolutional neural networks (CNN), long short-term memory networks (LSTM) and other deep learning models have shown excellent capabilities in processing complex data such as time series data and spatial data. However, since deep learning models usually have a large number of parameters and computational complexity, the model training process has a high demand for computing power. However, traditional single computing architecture / homogeneous computing architecture often has performance bottlenecks and low energy efficiency, and can no longer meet the needs of large-scale thermal big data computing and thermal large model training.

[0039] Taking the above problems into consideration, the exemplary embodiments of the present disclosure propose a computing power scheduling method for model training, which can be applied to any application scenario that requires large model training, especially to application scenarios that have high requirements for real-time model training. For example, in the thermal field, it is usually necessary to train a large thermal model to predict the heating load. However, in the process of training a large thermal model, the thermal data used usually have many abnormal values ​​or missing values, and are affected by many external factors. At the same time, the real-time requirements of thermal data are high, and it is necessary to ensure that the heating load can be predicted within the effective time of the thermal data, otherwise erroneous prediction results will be generated. Therefore, the process of training a large thermal model in the thermal field needs to ensure the computational efficiency of model training to ensure that the training and application of the model can be achieved within a shorter effective time interval.

[0040] The method is to divide the model to be trained into multiple first model subtasks in response to receiving a training task request for the model to be trained; for each target model subtask in the multiple first model subtasks, according to the pre-configured computing power mapping table, determine the heterogeneous computing power unit of the target resource type corresponding to the target model subtask from the heterogeneous computing power architecture; wherein the heterogeneous computing power architecture includes at least one computing node, the computing node is configured with at least one heterogeneous computing power unit, and the computing power mapping table contains a one-to-one correspondence between each computing task and the heterogeneous computing power unit of the corresponding resource type; determine the target heterogeneous computing power unit from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and assign the target model subtask to the target heterogeneous computing power unit to execute the target model subtask based on the target heterogeneous computing power unit. On the one hand, the method splits the model training task for the model to be trained into multiple first model subtasks so as to perform parallel task processing for multiple first model subtasks. Compared with the method of directly processing the model training task of the model to be trained in the related technical solution, the time consumption in the model training process is greatly reduced, thereby improving the efficiency of model training. On the other hand, the method allocates each first model subtask to a heterogeneous computing unit of the corresponding resource type through a pre-configured computing power mapping table. Since heterogeneous computing units of different resource types have different computing capabilities for different computing tasks, the above method can fully utilize the hardware resources of each heterogeneous computing unit and allocate the best performance heterogeneous computing unit for processing according to the specific model task requirements, thereby optimizing the hardware resource utilization in the system, improving the flexibility of model training task processing, and greatly reducing the time consumption of model training. Improve the efficiency of model training so as to better adapt to application fields with higher requirements for model training time.

[0041] The following will be combined Figure 1, a computing power scheduling method and device for model training proposed in this disclosure is applied to Figure 1 The embodiments are described in the system architecture of the exemplary application environment shown.

[0042] like Figure 1 As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. In this scenario, a terminal device 101 and multiple servers 102, 103, 104 and a scheduling server 105 may be included. Among them, multiple servers 102, 103, 104 and a scheduling server 105 are respectively used as computing nodes to form a heterogeneous computing power architecture, and servers 102, 103, 104 are configured with at least one heterogeneous computing power unit 1~n1, 1~n2, 1~n3 of the same or different computing resource types. For example, taking each server as an example of configuring a heterogeneous computing power unit of the same computing resource type, each server is equipped with 2 Intel Xeon CPUs, 4 NVIDIA A100 GPUs and 1 FPGA acceleration card. For another example, each server is configured with heterogeneous computing power units of different computing resource types, for example, 2 Intel Xeon CPUs, 4 NVIDIA A100 GPUs are configured in server 102, 4 NVIDIA A100 GPUs are configured in server 103, and 1 FPGA acceleration card is configured in server 104.

[0043] Among them, the terminal device 101 can be, for example, a mobile phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a smart car device, a smart wearable device, a smart TV, and an aircraft, etc., any device that involves the need to use the model to be trained to perform corresponding functions. The terminal device 101 can be installed with a target application, and the target application can have a display function of the model prediction results, etc. The application involved in the embodiment of the present application can be a software client, or it can be a client such as a web page or a small program. Servers 102, 103, and 104 are servers corresponding to software or web pages, small programs, etc., and do not limit the specific type of the client.

[0044] Servers 102, 103, 104 and scheduling server 105 are all backend servers of the target application, used to provide corresponding backend services for it, wherein scheduling server 105 is used to allocate computing resources for the model to be trained, and servers 102, 103, 104 provide corresponding computing resources for the model training tasks of the model to be trained. The above-mentioned servers 102, 103, 104 and scheduling server 105 can all be independent physical servers, or server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), as well as big data and artificial intelligence platforms, but are not limited to these.

[0045] In the embodiment of the present application, the terminal device 101 and the servers 102, 103, 104 and the dispatch server 105 can be directly or indirectly connected to each other through one or more networks 106. The network 106 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network. Of course, it can also be other possible networks, which are not limited in the embodiment of the present application. It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0046] For example, in an exemplary embodiment, a user triggers a corresponding model training / model application task request through a target application running in the terminal device 101, and the scheduling server 105 divides the model to be trained into multiple first model subtasks in response to receiving the training task request for the model to be trained; for each target model subtask in the multiple first model subtasks, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask is determined from the heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node (for example, servers 102, 103, 104), at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table contains a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; a target heterogeneous computing power unit is determined from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and the target model subtask is allocated to the target heterogeneous computing power unit to execute the target model subtask based on the target heterogeneous computing power unit.

[0047] In the following, the method provided by the exemplary embodiment of the present application is described with reference to the accompanying drawings in combination with the application scenarios described above. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in this respect. Figure 2 , and Figure 1 Taking the scheduling server 105 as an example of the execution subject, a computing power scheduling method for model training provided by the embodiment of the present disclosure is exemplarily described. Figure 2 As shown, it is a flow chart of a computing power scheduling method for model training provided in an embodiment of the present disclosure. The specific implementation process of the method is as follows: Step S201-Step S203: Step S201: In response to receiving a training task request for a model to be trained, dividing the model to be trained into multiple first model subtasks.

[0048] Step S202: for each target model subtask in the multiple first model subtasks, determine a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from the heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table contains a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type.

[0049] Step S203: determine the target heterogeneous computing unit from the heterogeneous computing units of the target resource type of each computing node in the heterogeneous computing architecture, and assign the target model subtask to the target heterogeneous computing unit to execute the target model subtask based on the target heterogeneous computing unit.

[0050] exist Figure 2In the provided technical solution, on the one hand, the method splits the model training task for the model to be trained into multiple first model subtasks, so as to perform parallel task processing for multiple first model subtasks. Compared with the method of directly processing the model training task for the model to be trained in the related technical solution, the time consumption in the model training process is greatly reduced, thereby improving the efficiency of model training. On the other hand, for each first model subtask, the method allocates each first model subtask to a heterogeneous computing unit of the corresponding resource type through a pre-configured computing power mapping table. Since the computing power of heterogeneous computing units of different resource types for different computing tasks is different, for example, GPU is better at processing image data (for example, processing image data faster and more accurately) than heterogeneous computing units of the CPU resource type. Therefore, the above method can make full use of the hardware resources of each heterogeneous computing unit, and allocate the best performance heterogeneous computing unit for processing according to the specific model task requirements, thereby optimizing the utilization of hardware resources in the system, improving the flexibility of model training task processing, and greatly reducing the time consumption of model training, improving the efficiency of model training, so as to better adapt to the application fields with higher requirements for model training time.

[0051] The following will be combined with specific embodiments to Figure 2 The specific implementation methods of each step in the embodiment shown are described in detail: In step S201, in response to receiving a training task request for a model to be trained, the model to be trained is divided into a plurality of first model subtasks.

[0052] Among them, the model to be trained can be any type of neural network model such as deep neural network DNN, convolutional neural network model CNN, long short-term memory network LSTM, etc. In this embodiment, the model to be trained can be a large model with a large number of model parameters and a certain computational complexity.

[0053] Exemplarily, a user may initiate a training task request for the model to be trained to the scheduling server through a terminal device, and the scheduling server, in response to receiving the training task request for the model to be trained, divides the model to be trained into multiple first model subtasks.

[0054] In an optional embodiment of the present disclosure, when executing step S201 to divide the model to be trained into multiple first model subtasks, the model to be trained can be divided according to preset parameters to obtain multiple first model subtasks.

[0055] The preset parameters include at least one or more of the following: Real-time demand parameters; Task magnitude parameters; Data type parameter; Parameters of idle computing resources of each heterogeneous computing unit.

[0056] The following is an exemplary description of an embodiment of the above preset parameters: In an optional embodiment, the scheduling server may divide the model to be trained according to the real-time requirement parameters. For example, the model to be trained may be divided into multiple first model subtasks such as those with extremely high real-time requirement (e.g., within 1 second), those with high real-time requirement (e.g., within 5 seconds), and those with low real-time requirement (e.g., no requirement).

[0057] In an optional embodiment, taking the task magnitude parameter as an example, the scheduling server may divide the model to be trained according to the task magnitude parameter, for example, it may be divided into a plurality of first model subtasks such as a lightweight task, a medium-weight task, and a heavy-weight task. It should be explained that the scheduling server may determine the task magnitude parameter according to the task processing amount / the amount of computation required by the heterogeneous computing power unit.

[0058] In an optional embodiment, taking data type parameters as an example, the scheduling server can divide the model to be trained according to the data type parameters, for example, it can be divided into multiple first model subtasks such as image data type, audio data type, matrix data type parameters, vector data type, etc.

[0059] In an optional embodiment, taking the idle computing resource parameters of each heterogeneous computing power unit in the heterogeneous computing power architecture as an example, the scheduling server determines the heterogeneous computing power units with sufficient idle computing resources by counting the idle computing resource parameters of each heterogeneous computing power unit in the heterogeneous computing power architecture, thereby dividing the tasks according to the heterogeneous computing power units with more idle computing resources to obtain multiple first model subtasks.

[0060] Among them, heterogeneous computing units can include CPU, GPU, FPGA, ASIC, tensor processing unit (TPU), neural network processor (NPU) and other hardware computing units. And each heterogeneous computing unit has a certain amount of computing resources. The computing resources include occupied computing resources and idle computing resources. The occupied computing resources are the resources that are executing task processing and can no longer be called for task processing, while the idle computing resources are the resources that are idle and can be called for task processing.

[0061] For example, the GPU occupancy rate in the current heterogeneous computing architecture is generally high (that is, the amount of idle computing resources is small), while the CPU occupancy rate is low (that is, the amount of idle computing resources is large). In this case, the model to be trained can be divided into more first model subtasks that can be processed by the CPU to improve resource utilization and task processing speed, further improving the model training speed.

[0062] It is understandable that the model to be trained can also be divided in a manner of combining the above two or more preset parameters to obtain multiple first model subtasks. For example, the task division can be performed according to a combination of a real-time requirement parameter and a task magnitude parameter. When multiple preset parameters are used for task division, the task division can be performed directly, or a weight value can be assigned to each preset parameter to perform task division based on the weight value. The embodiments of the present disclosure do not impose any special restrictions on this.

[0063] It should be explained that, in addition to the above embodiments, the model to be trained can also be divided according to the various steps included in the model training process or divided according to other dimensions. For example, for an image reconstruction model, the task to be trained can be divided into image feature extraction, image preprocessing, matrix calculation, projection reconstruction, three-dimensional information extraction, model optimization, parameter update, etc. based on the various steps included in the training process, and can be divided into the computing tasks corresponding to the above steps in turn. The disclosed embodiment does not make any exhaustive or limiting methods for dividing the model to be trained.

[0064] In this embodiment, the model to be trained is divided by at least one preset parameter, so that a large-scale model to be trained can be divided into first model subtasks of different task types / task characteristics, so that each first model subtask can be subsequently allocated to a heterogeneous computing power unit that matches its task characteristics according to the task characteristics of each first model subtask, thereby fully integrating and utilizing the heterogeneous computing power resources provided by each computing node, thereby reducing the time of model training.

[0065] In step S202, for each target model subtask in multiple first model subtasks, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask is determined from the heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table contains a one-to-one correspondence between each computing task and the heterogeneous computing power unit of the corresponding resource type.

[0066] Among them, the computing node is a device that carries various heterogeneous computing power units. For example, the computing node can be a server / terminal device that carries FPGA, CPU, GPU, etc., and the embodiments of the present disclosure do not impose any restrictions on this.

[0067] The computing task can be any task type that can be solved by each heterogeneous computing unit. In this embodiment, the computing task can be each first model subtask. The heterogeneous computing architecture includes at least one computing node, and at least one heterogeneous computing unit is configured in the computing node.

[0068] For example, when building a heterogeneous computing power architecture, a heterogeneous computing power unit of a corresponding resource type can be configured for each computing node according to the application scenario of the model to be trained. For example, if the model to be trained is an image reconstruction model, and the model training process mostly involves the processing of image data, then when building a heterogeneous computing power architecture, more GPUs can be configured for each computing node, or computing nodes configured with more GPUs can be used.

[0069] It should be explained that when configuring more GPUs for each computing node, more GPUs can be added to a single computing node, or multiple computing nodes can be used as a group of computing nodes to add more GPUs to the computing node. The embodiments of the present disclosure do not impose any restrictions on this.

[0070] For the above situation where multiple computing nodes are used as a group of computing nodes, the scheduling server can implement parallel processing of multiple computing nodes in the group according to task processing requirements to further improve the model training efficiency.

[0071] Exemplarily, for each target model subtask, the heterogeneous computing power unit of the target resource type corresponding to the target model subtask can be determined according to the pre-configured computing power mapping table. In this embodiment, heterogeneous computing power units of different resource types have different hardware characteristics. For example, the hardware characteristics of GPU or TPU are that they can process image data and matrix calculations in model training faster; CPU can process lightweight tasks quickly; for the first model subtask with higher real-time requirements, hardware acceleration processing can be performed through FPGA, etc. Of course, multiple heterogeneous computing power units can also be used to process a computing task simultaneously, such as through GPU-CPU hybrid computing, to achieve efficient collaboration of data parallel computing and model parallel computing.

[0072] When executing step S202 to determine the heterogeneous computing power unit of the target resource type corresponding to the target model subtask from the heterogeneous computing power architecture according to the pre-configured computing power mapping table, the computing power mapping table can be pre-built. Figure 3 The process of constructing a computing power mapping table is illustrated.

[0073] In an alternative embodiment, Figure 3 A flowchart schematically shows a method for constructing a computing power mapping table in this exemplary embodiment; Figure 3As shown, the method for constructing a computing power mapping table may include the following steps S301 to S302: Step S301: for each target computing task in each computing task, determine the computing speed of the heterogeneous computing power units of each resource type to execute the target computing task.

[0074] Step S3021: Determine the heterogeneous computing power unit of the resource type with the fastest computing speed as the heterogeneous computing power unit of the resource type that matches the target computing task, so as to construct a computing power mapping table.

[0075] For example, when constructing a computing power mapping table, in order to improve the efficiency of the model training task, for each target computing task, heterogeneous computing power units of various resource types can be used to respectively execute the target computing task, so as to count the computing speed of the heterogeneous computing power units of various resource types in executing the target computing task, and thus use the heterogeneous computing power unit with the fastest computing speed as the resource type that matches the target computing task. In this way, each computing task can be matched to a heterogeneous computing power unit of the corresponding resource type, thereby constructing a computing power mapping table.

[0076] For example, suppose there is a computing task A. The computing nodes in the heterogeneous computing architecture contain heterogeneous computing units, namely, heterogeneous computing unit a, heterogeneous computing unit b, and heterogeneous computing unit c. The computing task A is executed by heterogeneous computing unit a, heterogeneous computing unit b, and heterogeneous computing unit c. Assuming that heterogeneous computing unit b has the fastest computing speed in executing computing task A, heterogeneous computing unit b is used as the heterogeneous computing unit matching computing task A, thereby constructing a computing power mapping table.

[0077] In this embodiment, by determining the heterogeneous computing power unit of the resource type with the fastest computing speed as the heterogeneous computing power unit of the resource type that matches the target computing task, it is convenient to allocate each first model subtask to the heterogeneous computing power unit with the fastest computing speed during subsequent calculation, thereby reducing the time consumption of the first model subtask and achieving the effect of improving the processing efficiency of the first model subtask.

[0078] In addition to the above embodiments, since each heterogeneous computing power unit will process each computing task in actual applications, if the current load of the heterogeneous computing power unit matching the computing task is high, then the computing task will need to be queued when assigned to the heterogeneous computing power unit to wait for the current load of the heterogeneous computing power unit to decrease, which will seriously affect the processing efficiency of the first model subtask.

[0079] In order to improve the processing efficiency of the first model subtask, when constructing the computing power mapping table, you can refer to Figure 4As shown, execute step S301, determine the computing speed of the heterogeneous computing power units of each resource type to execute the target computing task for each target computing task in each computing task; then execute step S3022, sort in descending order of computing speed; finally, execute step S303, match the heterogeneous computing power units of resource types in descending order of computing speed to construct a computing power mapping table.

[0080] In this embodiment, the computing speed can be sorted from high to low, so that the heterogeneous computing power units of resource types can be matched in sequence from high to low computing speed to construct a computing power mapping table.

[0081] Similarly, assume a computing task A, and the computing nodes in the heterogeneous computing power architecture contain heterogeneous computing power units, namely, heterogeneous computing power unit a, heterogeneous computing power unit b, and heterogeneous computing power unit c; computing task A is executed by heterogeneous computing power unit a, heterogeneous computing power unit b, and heterogeneous computing power unit c, respectively. Assuming that heterogeneous computing power unit b has the fastest computing speed in executing computing task A, heterogeneous computing power unit a is the second fastest, and heterogeneous computing power unit c is the slowest, then in order of computing speed from high to low, heterogeneous computing power unit b, heterogeneous computing power unit a, and heterogeneous computing power unit c are sequentially used as heterogeneous computing power units matching computing task A, thereby constructing a computing power mapping table.

[0082] In this embodiment, when the current load of heterogeneous computing power unit b is high and cannot receive computing task A and process it quickly, computing task A can be assigned to heterogeneous computing power unit a. Similarly, if the current load of the heterogeneous computing power unit is high and cannot receive computing task A and process it quickly, it can be assigned to heterogeneous computing power unit c.

[0083] Through the embodiments of the present disclosure, it is possible to ensure that computing tasks can be processed quickly, thereby improving model training efficiency.

[0084] In an optional embodiment, in addition to the constructed heterogeneous computing power architecture, other parallel computing power units can be configured for the above-constructed heterogeneous computing power architecture to add additional computing power to supplement each first model subtask in the model to be trained, so as to further accelerate the computational efficiency of model task processing.

[0085] In an optional embodiment of the present disclosure, multiple first model subtasks include at least a model building task. The method can also obtain a target training data set; determine data features of each training data contained in the target training data set; construct a model structure of the model to be trained according to the data features to execute the model building task, and determine a heterogeneous computing power unit of a target resource type corresponding to the model building task from the heterogeneous computing power architecture.

[0086] Exemplarily, the plurality of first model subtasks include at least a model building task. During the execution of the model building task, the model structure of the to-be-trained model may be determined according to the data features of each training data included in the target training data set. That is, the model structure is customized according to the data characteristics and requirements, and the corresponding algorithm model structure is selected and built.

[0087] In an optional embodiment of the present disclosure, the model structure of the model to be trained includes at least one or more of the following: a trend analysis model structure, a regression analysis model structure, a time series model structure, a machine learning model structure, a mechanism and data fusion model structure, and a grey system model structure.

[0088] The following will take thermal data in the field of thermal application scenarios as an example. The target training data set contains collected historical heating data, such as temperature, humidity, meteorological conditions, building characteristics, user needs and other data.

[0089] 1) For the trend analysis model structure: a. The trend analysis model structure can fit a curve based on historical training data to reflect the growth trend of the load.

[0090] b. Commonly used trend models include linear trend models, polynomial trend models, logarithmic trend models, etc.

[0091] c. This type of model usually does not consider random errors when processing historical data and fitting curves, and is suitable for predictions with large samples and consistent historical development patterns.

[0092] 2) Regression analysis model structure: a. Regression analysis makes predictions by establishing a functional relationship between influencing factors (such as outdoor temperature, air temperature, population, etc.) and heating load.

[0093] b. The regression analysis model structure requires a large amount of historical data and influencing factor data, and is suitable for scenarios with abundant data and a stable relationship between influencing factors and loads.

[0094] 3) Time series model structure: a. The time series method uses the time series characteristics of load data, such as seasonality and periodicity, to establish a prediction model.

[0095] b. Common time series models include the autoregressive model (AR model), the moving average (MA) model and their combination models, such as the autoregressive moving average model (ARMA), the autoregressive integrated moving average model (ARIMA), etc.

[0096] 4) Machine Learning Model Structure: a. Machine learning algorithms, such as support vector machine (SVM), neural network, random forest, etc., are also widely used in load forecasting.

[0097] b. The machine learning model structure can automatically learn and extract features from historical data, model complex nonlinear relationships, and has high prediction accuracy.

[0098] 5) Mechanism and data fusion model structure: a. The mechanism and data fusion model structure combines the advantages of the mechanism model and the data-driven model, aiming to solve the problem of incomplete or low-quality data.

[0099] b. Mechanistic models provide support for physical equations and thermodynamic principles, while data-driven models use historical data for training and learning.

[0100] c. By integrating these two methods, the stability and accuracy of prediction can be improved, which is particularly suitable for scenarios with unfavorable data conditions.

[0101] 6) Grey system model structure: a. Grey system theory is applicable to analysis and prediction under conditions with less information. It treats the load sequence as a grey system.

[0102] b. By generating transformations to establish a prediction model, the grey model can provide more reliable prediction results when the data is incomplete or uncertain.

[0103] In this regard, when constructing the model structure of the model to be trained, real-time matching can be performed based on the actual application scenario of the model to be trained and the data features of each training data contained in the collected target training data set to find a model structure with a high degree of matching. In addition, it should be explained that in the process of constructing the model structure, the input layer and output layer of the model need to be designed to ensure that the model can receive each training data in the target training data set and output the model prediction results; and configure the corresponding model parameters, such as learning rate, batch size, etc., so as to perform subsequent model optimization tasks.

[0104] For example, when receiving a model building task, you can select a suitable model structure for training, such as time series prediction models such as LSTM and GRU, or models based on the Transformer structure. You can also apply strategies such as early stopping and learning rate scheduling to prevent overfitting during model training and accelerate convergence.

[0105] Furthermore, engineering techniques such as mixed precision training, gradient accumulation, automatic differentiation library optimization, and cache optimization can be introduced to reduce memory usage and bandwidth requirements.

[0106] Taking the construction of a large thermal model as an example, a convolutional neural network model (CNN) is selected as the model structure, and the input layer and output layer are designed to receive thermal data and output prediction results. The model parameters are configured, such as a learning rate of 0.001 and a batch size of 64; or an LSTM neural network is used for training of heating load prediction, and batch gradient descent is used for optimization. GPUs are used to accelerate matrix calculations during training, and TPUs are used to accelerate gradient calculations in the back-propagation algorithm.

[0107] For the model training task, it includes collecting training data sets. Since there are various problems with the data collected in the training data sets, in order to further improve the accuracy of the model training results, data preprocessing can be performed based on the collected data sets.

[0108] In an optional embodiment of the present disclosure, multiple first model subtasks include data preprocessing tasks, and the scheduling server obtains an initial training data set; performs preprocessing operations on each training data in the initial training data set to obtain a preprocessed target training data set; wherein, data cleaning processing, data denoising processing, normalization processing, data standardization processing, data enhancement processing, and missing value filling processing are performed.

[0109] For example, in the field of thermal power, a large thermal model is trained to predict the heating load. During the training of the large thermal model, the thermal data used usually has many abnormal values ​​or missing values, and is affected by many external factors. Therefore, it is necessary to preprocess the training data in the initial training data set before model training.

[0110] For example, each training data in the initial training data set is subjected to data cleaning, data denoising, normalization, data standardization, data enhancement, missing value filling, etc. to ensure data quality and further improve the accuracy of the model prediction results.

[0111] It should be explained that the training data in the target training data set can be divided into training set, validation set and test set, where the training set is the data sample used for model fitting. The validation set is a set of samples set aside during the model training process. It can be used to adjust the model's hyperparameters and to make a preliminary assessment of the model's capabilities. It is usually used to verify the current model's generalization ability (accuracy, recall, etc.) during model iterative training to decide whether to stop training. The test set is used to evaluate the generalization ability of the final model, but it cannot be used as a basis for algorithm-related choices such as parameter adjustment and feature selection.

[0112] In an optional embodiment, if the data volume of the target training data set is greater than a data volume threshold, the target training data set is distributedly stored.

[0113] For example, if the target training data set has a large amount of data, in order to improve resource utilization, a distributed storage technology (such as Hadoop Distributed File System (HDFS), Ceph, etc.) can be used for data management.

[0114] In an optional embodiment, the plurality of first model subtasks may further include a feature extraction task. When executing the feature extraction task, in addition to using hardware acceleration, a suitable feature extraction method may also be selected for each data type.

[0115] For example, based on expert experience or automated feature selection methods, the most influential features for heating load forecasting can be extracted. Lag feature generation and sliding window processing can be performed on time series data. If there are multiple sensor data or image data (such as building heat load data), feature extraction can be performed using methods such as convolutional neural network models (CNN).

[0116] In step S203, the target heterogeneous computing unit is determined from the heterogeneous computing units of the target resource type of each computing node in the heterogeneous computing architecture, and the target model subtask is assigned to the target heterogeneous computing unit to execute the target model subtask based on the target heterogeneous computing unit.

[0117] For example, by counting the current load of the heterogeneous computing units of the target resource type of each computing node, the target model subtask can be preferentially allocated to the heterogeneous computing unit with less current load, so as to determine it as the target heterogeneous computing unit.

[0118] In an optional embodiment of the present disclosure, when the target heterogeneous computing power unit is in multiple computing nodes in the heterogeneous computing power architecture, that is, the target model subtask can be allocated to multiple heterogeneous computing power units in the multiple computing nodes, then in the step of determining the target heterogeneous computing power unit from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and allocating the target model subtask to the target heterogeneous computing power unit, the scheduling server can determine whether the target heterogeneous computing power unit is in the multiple computing nodes in the heterogeneous computing power architecture. If the target heterogeneous computing power unit is in the multiple computing nodes in the heterogeneous computing power architecture, the target model subtask is divided into multiple second model subtasks to allocate the multiple second model subtasks to the target heterogeneous computing power unit in the multiple computing nodes.

[0119] By further dividing the target model subtasks, multiple target heterogeneous computing units in multiple computing nodes can be used for parallel computing, thereby realizing large-scale distributed training of the model, such as data parallelism, model parallelism, pipeline parallelism and other methods, to improve the model training speed.

[0120] This embodiment realizes the dynamic allocation and scheduling of hardware resources of heterogeneous computing units, ensuring that resources are used on demand at different stages of model training, thereby improving the utilization rate of hardware resources.

[0121] In an optional embodiment of the present disclosure, the multiple first model subtasks include at least a third model subtask, and the scheduling server may also execute a fourth model subtask among the multiple first model subtasks based on the heterogeneous computing power units of each computing node in the heterogeneous computing power architecture to generate a target training model; determine a model prediction result based on the target training model; in response to a loss value between the model prediction result and the actual result being greater than a preset loss value, execute the third model subtask to update the target training model based on the third model subtask until the loss value between the model prediction result and the actual result of the updated target training model is no greater than the preset loss value.

[0122] Among them, the fourth model subtask is the first model subtask executed before the third model subtask; the loss value can be characterized by evaluation indicators such as prediction accuracy, error rate, and R² of the model prediction results. It should be explained that R² is a commonly used regression model evaluation indicator that can characterize the degree of model fit.

[0123] Exemplarily, the plurality of first model subtasks further include a third model subtask, wherein the third model subtask may be a model optimization task, and correspondingly, the fourth model subtask may be the third model subtask, that is, a set of all subtasks for building a target training model before the model optimization task. After the heterogeneous computing power units based on the computing nodes in the heterogeneous computing power architecture have completed all the fourth model subtasks before the model optimization task, a target training model can be generated. For example, the target training model is a large thermal model, and the model prediction result may be a heating load prediction. So that the third model subtask, that is, the model optimization task, can be subsequently executed for the target training model.

[0124] In order to continuously improve the applicability and accuracy of the target training model, the model prediction results can be obtained based on the generated target training model, and the model prediction results can be compared with the actual results to obtain the loss value between the model prediction results and the actual results. If the loss value is greater than the preset loss value, it is considered that the model effect is not good and needs to be adjusted until the loss value between the model prediction results and the actual results of the updated target training model is no greater than the preset loss value.

[0125] In an optional embodiment of the present disclosure, when updating the target training model, update optimization may be performed based on at least one of the following: 1) Adjust the model structure, such as increasing or decreasing the number of network layers or nodes.

[0126] 2) Optimize hyperparameters (such as learning rate, batch size, etc.).

[0127] 3) Perform more feature selection or enhancement on the data.

[0128] 4) Adjust computing resource allocation strategy to improve computing efficiency.

[0129] In addition to the above embodiments, the training process can also be monitored during the model training process of the target training model to evaluate and debug the model performance. For example, different regularization techniques, learning rate scheduling, and loss functions can be tried to optimize the model.

[0130] In addition to the above embodiments, the model can be further optimized through model pruning and quantization. For example, the trained model is pruned and quantized to remove redundant parameters and layers in the model, and the parameters are converted from floating point numbers to fixed point numbers. Through pruning and quantization operations, the model size is reduced and the computing resources required for reasoning are reduced.

[0131] It should be explained that after the target training model is generated, it is also necessary to deploy the model to the actual application scenario. For example, in a city heating system, the trained target training model can be deployed to the actual heating system and predictions can be updated regularly. In addition, the model prediction results can be fed back in real time so that the system can be adjusted according to the actual situation and the heating strategy can be optimized.

[0132] Exemplarily, when the target training model is deployed to an actual application scenario, containerization technology (such as Docker) can be used to encapsulate it into a portable container for easy deployment in different environments. In addition, the allocation and scheduling of intelligent computing resources in the heterogeneous intelligent computing resource layer can also be achieved through the Kubernetes module, specifically including: the scheduling server allocates intelligent computing resources of the heterogeneous intelligent computing resource layer to the model training task according to the settings of the model training task and the resource topology generation information, and then calls the Kubernetes module to create a container for the model training task, and performs resource scheduling in units of containers. In addition, in actual applications, the target training model is called for prediction through the application programming interface (Application Programming Interface, API) or service framework (such as TensorFlow Serving, FastAPI framework, etc.).

[0133] In order to improve the interactivity of the model, the method can also be integrated into the application platform for visualization and support for real-time user operation. Taking the heating system as an example, the computing power scheduling method for model training provided by the embodiment of the present disclosure can be integrated with the existing heating system platform (such as heat exchange station automation system, energy management platform, smart heating system), so as to make predictions through API calls to the model, and realize the automation of data collection, model training, prediction analysis and control scheduling. In addition, it is also possible to combine real-time meteorological data to predict future loads, and give optimized heating scheduling suggestions based on the prediction results to help dispatchers optimize heat source configuration.

[0134] In order to implement the computing power scheduling method for the above-mentioned model training, a computing power scheduling device for model training is provided in one embodiment of the present disclosure. Figure 5 The schematic diagram shows the schematic architecture of the computing power scheduling device for model training.

[0135] Among them, the computing power scheduling device 500 for model training includes a task division module 501, a computing power determination module 502 and a task allocation module 503.

[0136] The task division module 501 is used to divide the model to be trained into multiple first model subtasks in response to receiving a training task request for the model to be trained; the computing power determination module 502 is used to determine, for each target model subtask in the multiple first model subtasks, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from the heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table includes a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; the task allocation module 503 is used to determine a target heterogeneous computing power unit from the heterogeneous computing power units of the target resource type of each computing node in the heterogeneous computing power architecture, and allocate the target model subtask to the target heterogeneous computing power unit, so as to execute the target model subtask based on the target heterogeneous computing power unit.

[0137] In an optional embodiment of the present disclosure, the task division module 501 is specifically used to divide the model to be trained according to preset parameters to obtain multiple first model subtasks; wherein the preset parameters include at least one or more of the following: Real-time demand parameters; Task magnitude parameters; Data type parameter; Parameters of idle computing resources of each heterogeneous computing unit.

[0138] In an optional embodiment of the present disclosure, the device may also include a computing speed determination module and a mapping table construction module, wherein the computing speed determination module is used to determine the computing speed of heterogeneous computing power units of each resource type to execute the target computing task for each target computing task in each computing task; the mapping table construction module is used to determine the heterogeneous computing power unit of the resource type with the fastest computing speed as the heterogeneous computing power unit of the resource type matching the target computing task, so as to construct a computing power mapping table.

[0139] In an optional embodiment of the present disclosure, multiple first model subtasks include at least a model building task, and the device may also include a data acquisition module, a data feature determination module and a task execution module; the data acquisition module is used to acquire a target training data set; the data feature determination module is used to determine the data features of each training data contained in the target training data set; the task execution module is used to construct a model structure of the model to be trained according to the data features to execute the model building task, and determine the heterogeneous computing power unit of the target resource type corresponding to the model building task from the heterogeneous computing power architecture; wherein the model structure of the model to be trained includes at least one or more of the following: trend analysis model structure, regression analysis model structure, time series model structure, machine learning model structure, mechanism and data fusion model structure, and grey system model structure.

[0140] In an optional embodiment of the present disclosure, the data acquisition module is used to perform preprocessing operations on each training data in the initial training data set to obtain a preprocessed target training data set; wherein the preprocessing operation includes at least one or more of the following: data cleaning processing, data denoising processing, normalization processing, data standardization processing, data enhancement processing, and missing value filling processing.

[0141] In an optional embodiment of the present disclosure, the task allocation module 503 is used to divide the target model subtask into multiple second target model subtasks when the target heterogeneous computing power unit is in multiple computing nodes in the heterogeneous computing power architecture, so as to allocate the multiple second target model subtasks to the target heterogeneous computing power unit in the multiple computing nodes.

[0142] In an optional embodiment of the present disclosure, the first model subtask includes at least a third model subtask, and the device further includes a model generation module and a model update module. The model generation module is used to execute a fourth model subtask among multiple first model subtasks based on the heterogeneous computing power units of each computing node in the heterogeneous computing power architecture to generate a target training model; the fourth model subtask is a model subtask executed before the third model subtask; according to the target training model, a model prediction result is determined; and the model update module is used to execute the third model subtask in response to a loss value between the model prediction result and the actual result being greater than a preset loss value, so as to update the target training model based on the third model subtask until the loss value between the model prediction result and the actual result of the updated target training model is no greater than the preset loss value.

[0143] The computing power scheduling device 500 for model training provided in the embodiments of the present disclosure can execute the technical solution of the computing power scheduling method for model training in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the computing power scheduling method for model training. Please refer to the implementation principle and beneficial effects of the computing power scheduling method for model training, which will not be repeated here.

[0144] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0145] The program product for implementing the above method according to an embodiment of the present invention may adopt a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0146] The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0147] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0148] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.

[0149] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0150] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0151] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0152] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0153] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: the at least one processing unit 610, the at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), and a display unit 640.

[0154] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 610 can perform the following steps: Figure 2 Steps S201 to S203 shown in FIG.

[0155] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0156] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0157] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0158] The electronic device 600 may also communicate with one or more external devices 1000 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems, etc.

[0159] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0160] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0161] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0163] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A computing power scheduling method for model training, characterized in that: include: In response to receiving a training task request for a model to be trained, dividing the model to be trained into a plurality of first model subtasks; For each target model subtask in the multiple first model subtasks, determine a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from a heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table includes a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; Determine a target heterogeneous computing unit from the heterogeneous computing units of the target resource type of each computing node in the heterogeneous computing architecture, and assign the target model subtask to the target heterogeneous computing unit to execute the target model subtask based on the target heterogeneous computing unit.

2. The method according to claim 1, characterized in that The step of dividing the model to be trained into a plurality of first model subtasks includes: Dividing the model to be trained according to preset parameters to obtain the multiple first model subtasks; The preset parameters include at least one or more of the following: Real-time demand parameters; Task magnitude parameters; Data type parameter; Parameters of idle computing resources of each heterogeneous computing unit.

3. The method according to claim 1, characterized in that: The method further comprises: For each target computing task in each computing task, determining the computing speed of the heterogeneous computing units of each resource type to execute the target computing task; The heterogeneous computing power unit of the resource type with the fastest computing speed is determined as the heterogeneous computing power unit of the resource type matching the target computing task, so as to construct the computing power mapping table.

4. The method according to claim 1, characterized in that: The plurality of first model subtasks at least include a model building task, and the method further includes: Obtain the target training data set; Determining data features of each training data in the target training data set; Constructing a model structure of the model to be trained according to the data features to execute the model building task, and determining a heterogeneous computing power unit of a target resource type corresponding to the model building task from a heterogeneous computing power architecture; Among them, the model structure of the model to be trained includes at least one or more of the following: trend analysis model structure, regression analysis model structure, time series model structure, machine learning model structure, mechanism and data fusion model structure, and grey system model structure.

5. The method according to claim 4, characterized in that The step of obtaining a target training data set includes: Get the initial training data set; A preprocessing operation is performed on each training data in the initial training data set to obtain the target training data set; wherein the preprocessing operation includes at least one of the following: data cleaning processing, data denoising processing, normalization processing, data standardization processing, data enhancement processing, and missing value filling processing.

6. The method according to claim 1, characterized in that The step of determining a target heterogeneous computing unit from heterogeneous computing units of a target resource type of each computing node in the heterogeneous computing architecture, and allocating the target model subtask to the target heterogeneous computing unit includes: If the target heterogeneous computing power unit is in multiple computing nodes in the heterogeneous computing power architecture, the target model subtask is divided into multiple second model subtasks to allocate the multiple second model subtasks to the target heterogeneous computing power unit in the multiple computing nodes.

7. The method according to claim 1, characterized in that The plurality of first model subtasks include at least a third model subtask, and the method further includes: The heterogeneous computing power units of each computing node in the heterogeneous computing power architecture execute a fourth model subtask among the multiple first model subtasks to generate a target training model; the fourth model subtask is the first model subtask executed before the third model subtask; Training a model according to the target and determining a model prediction result; In response to the loss value between the model prediction result and the actual result being greater than a preset loss value, the third model subtask is executed to update the target training model based on the third model subtask until the loss value between the model prediction result and the actual result of the updated target training model is no greater than the preset loss value.

8. A computing power scheduling device for model training, characterized in that: include: A task division module, configured to, in response to receiving a training task request for a model to be trained, divide the model to be trained into a plurality of first model subtasks; A computing power determination module is used to determine, for each target model subtask in the multiple first model subtasks, a heterogeneous computing power unit of a target resource type corresponding to the target model subtask from a heterogeneous computing power architecture according to a pre-configured computing power mapping table; wherein the heterogeneous computing power architecture includes at least one computing node, at least one heterogeneous computing power unit is configured in the computing node, and the computing power mapping table includes a one-to-one correspondence between each computing task and a heterogeneous computing power unit of a corresponding resource type; A task allocation module is used to determine a target heterogeneous computing unit from the heterogeneous computing units of the target resource type of each computing node in the heterogeneous computing architecture, and allocate the target model subtask to the target heterogeneous computing unit to execute the target model subtask based on the target heterogeneous computing unit.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computing power scheduling method for model training described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the computing power scheduling method for model training described in any one of claims 1 to 7 by executing the executable instructions.

Images