Resource scheduling method and model training method of large model

Abstract

The invention discloses a large model resource scheduling method and a model training method. The method relates to the technical field of artificial intelligence, and comprises the following steps: obtaining load historical data of a plurality of computing nodes corresponding to a large model in a historical time period, and determining current operation resources of the large model; inputting the load historical data into a load prediction model, and predicting the loads of the plurality of computing nodes in a preset time period by using the load prediction model to obtain load prediction data; determining deployment results of model instances deployed in the plurality of computing nodes based on the load prediction data; and performing resource scheduling based on the deployment result and the current running resource. Through the method and the device, the technical problem of low resource utilization rate of the large model caused by low resource scheduling accuracy of the large model in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a resource scheduling method and a model training method for a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, people are paying more and more attention to the research in multiple artificial intelligence research fields such as machine learning, natural language processing, computer vision, robotics, and expert systems. As a model with a large number of parameters in the fields of artificial intelligence machine learning and deep learning, the large model needs to consume a lot of resources when running. Therefore, resource scheduling of large models is one of the main means to ensure the resource utilization of large models, and resource scheduling of large models is also one of the main means to ensure the stable operation of large models. However, the accuracy of resource scheduling of large models in related technologies is low, resulting in low resource utilization of large models, which in turn leads to high cost of large model operation and maintenance and poor user experience.

[0003] Currently, no effective solution has been proposed for the technical problem of low resource utilization of large models due to low accuracy of resource scheduling of large models in related technologies. Summary of the invention

[0004] The main purpose of this application is to provide a resource scheduling method and a model training method for a large model, so as to solve the technical problem of low resource utilization of large models caused by low accuracy of resource scheduling of large models in related technologies.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a resource scheduling method for a large model is provided. The method includes: obtaining the load history data of multiple computing nodes corresponding to the large model in the historical time period, and determining the current running resources of the large model, wherein the model instances of the large model are deployed in the multiple computing nodes, and the model instances are used to represent the running copies of the large model; inputting the load history data into the load prediction model, using the load prediction model to predict the load of multiple computing nodes in the preset time period, and obtaining the load prediction data; determining the deployment results of the model instances deployed in the multiple computing nodes based on the load prediction data, wherein the deployment results are used to indicate whether to adjust the model instances; and performing resource scheduling based on the deployment results and the current running resources.

[0006] Further, the deployment results of the model instances deployed in multiple computing nodes are determined based on the load prediction data, including: in response to the load prediction data being greater than a first preset threshold, or, in response to the load prediction data being less than a second preset threshold, determining that the deployment result is to adjust the model instance; in response to the load prediction data being less than or equal to the first preset threshold, and the load prediction data being greater than or equal to the second preset threshold, determining that the deployment result is to prohibit adjustment of the model instance.

[0007] Furthermore, resources are scheduled for the large model based on the deployment results and current running resources, including: in response to the deployment result of adjusting the model instance, the current running resources are adjusted based on the load forecast data and the load history data to obtain the target running resources, and resources are scheduled according to the target running resources; in response to the deployment result of prohibiting the adjustment of the model instance, resources are scheduled based on the current running resources.

[0008] Furthermore, the current operating resources are adjusted based on the load prediction data and the load history data to obtain the target operating resources, including: in response to the load prediction data being greater than the load history data, determining a first number of model instances to be added, retrieving first operating resources corresponding to the first number from the resource pool, and determining the target operating resources based on the sum of the current operating resources and the first operating resources; in response to the load prediction data being less than the load history data, determining a second number of model instances to be reduced, returning second operating resources corresponding to the second number to the resource pool, and determining the target operating resources based on the difference between the current operating resources and the second operating resources.

[0009] Furthermore, resource scheduling is performed according to the target operating resources, including: at a preset time point before a preset time period, the target operating resources are allocated to the model instance.

[0010] Furthermore, the method also includes: obtaining sample load history data of a sample historical time period and sample load actual data of a sample preset time period; inputting the sample load history data into an initial load prediction model, and using the initial load prediction model to predict sample load data of multiple computing nodes in the sample preset time period; training the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0011] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a model training method is provided. The method comprises: obtaining sample load history data of a sample history time period and sample load actual data of a sample preset time period; inputting the sample load history data into an initial load prediction model, and using the initial load prediction model to obtain sample load prediction data of multiple computing nodes in the sample preset time period; training the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0012] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0013] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0014] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0015] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0016] In an embodiment of the present application, the load history data of multiple computing nodes corresponding to the large model in the historical time period is obtained, and the current running resources of the large model are determined; the load history data is input into the load prediction model, and the load prediction model is used to predict the load of multiple computing nodes in the preset time period to obtain load prediction data; the deployment results of the model instances deployed in the multiple computing nodes are determined based on the load prediction data; and resource scheduling is performed based on the deployment results and the current running resources. In the present application, the load prediction model is set to predict the load of multiple computing nodes of the large model in the preset time period, and the deployment results of whether the model instances are adjusted in the multiple computing nodes of the large model are determined according to the predicted load prediction data, and the large model is accurately scheduled for resources according to the deployment results and the current running resources, so as to achieve the purpose of effectively improving the accuracy of the resource scheduling of the large model, thereby achieving the technical effect of effectively improving the resource utilization of the large model, and then solving the technical problem of low resource utilization of the large model caused by the low accuracy of resource scheduling of the large model in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a computer terminal for implementing a resource scheduling method for a large model is shown;

[0019] Figure 2 is a flow chart of an optional large model resource scheduling method according to an embodiment of the present invention;

[0020] Figure 3 is a schematic structural diagram of an optional large-model resource scheduling device according to an embodiment of the present invention;

[0021] Figure 4 is a schematic structural diagram of another optional large-model resource scheduling device according to an embodiment of the present invention;

[0022] Figure 5 is a flowchart of an optional model training method according to an embodiment of the present invention;

[0023] Figure 6 is a schematic structural diagram of an optional model training device according to an embodiment of the present invention;

[0024] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0028] Example 1

[0029] According to an embodiment of the present application, an embodiment of a resource scheduling method for a large model is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a resource scheduling method for a large model is shown. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the resource scheduling method of the large model in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the resource scheduling method of the large model described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0035] Under the above operating environment, this application provides Figure 2 Resource scheduling method for the large model shown. Figure 2 is a flow chart of an optional large model resource scheduling method according to an embodiment of the present invention, such as Figure 2 As shown, the method comprises the following steps:

[0036] Step S102, obtain the load history data of multiple computing nodes corresponding to the big model in the historical time period, and determine the current running resources of the big model, wherein model instances of the big model are deployed in the multiple computing nodes, and the model instances are used to represent the running copies of the big model.

[0037] The above-mentioned large model refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called the foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large-scale language models (LLM) and multi-modal pre-training models.

[0038] It should be noted that when the large model is actually used, the pre-trained model can be fine-tuned by a small number of samples so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (Natural Language Processing, referred to as NLP), computer vision, speech processing and other fields, and can be specifically applied to computer vision tasks such as visual question answering (Visual Question Answering, referred to as VQA), image description (Image Caption, referred to as IC), image generation, etc., and can also be widely used in text-based sentiment classification, text summary generation, machine translation and other natural language processing tasks. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In this application, it is necessary to schedule the resources of the above-mentioned large model to improve the resource utilization of the above-mentioned large model and ensure the operating efficiency of the above-mentioned large model.

[0039] The above-mentioned computing nodes refer to the computing nodes of the large model, and the computing nodes of the above-mentioned large model are used to perform a single computer or server for computing tasks. In the field of deep learning and artificial intelligence, large models generally refer to complex models that require a large amount of computing resources, such as large neural networks. These models cannot be efficiently trained on a single computer because of their numerous parameters, so they need to be distributed on multiple computing nodes. In this application, it is necessary to collect the load historical data of multiple computing nodes of the large model in the historical time period.

[0040] The above-mentioned historical time period refers to the time period in which the big model has been running in history, wherein the length of the above-mentioned historical time period can be set according to the state of the big model, the length of the above-mentioned historical time period can also be set manually according to experience and needs, and the length of the above-mentioned historical time period can also be set according to the actual application scenario.

[0041] The above-mentioned load history data refers to the load data of multiple computing nodes of the above-mentioned large model in the historical time period, and the above-mentioned load refers to the workload undertaken by a single computing node in a distributed computing environment. In deep learning, machine learning or other tasks that require a large amount of computing resources, large models usually require multiple computing nodes to work together to complete training or reasoning tasks. Therefore, when the above-mentioned large model is running in the historical time period, the multiple computing nodes of the above-mentioned large model will generate the above-mentioned load history data in the above-mentioned historical time period. Among them, the above-mentioned load history data includes but is not limited to: CPU utilization, memory utilization, disk input and output rate, network input and output rate, temperature and energy consumption, etc.

[0042] The above-mentioned operating resources refer to the resources occupied by the current large model during operation. The above-mentioned large model needs to occupy a large amount of hardware resources during operation to ensure the stable operation of the above-mentioned large model. Among them, the above-mentioned operating resources include but are not limited to: central processing unit resources, memory resources, disk resources, network resources or energy consumption resources, etc.

[0043] The above-mentioned model instance refers to a specific model instance running in a specific hardware or software environment. The above-mentioned model instance refers to an instance that specifically performs the computing functions of multiple computing nodes of the above-mentioned large model. Among them, the above-mentioned model instance is a model running in a specific environment in the above-mentioned large model. The resource inclination of the above-mentioned computing nodes can be controlled by increasing or decreasing the model instances deployed in the above-mentioned multiple computing nodes.

[0044] The above-mentioned running copy refers to the running copy of the large model (Running Instance of Large Models, referred to as RILM), which refers to the instance of the actual operation and calculation of the large machine learning model in a specific hardware or software environment. These models may include deep learning models, natural language processing models, computer vision models, etc., which usually require a large amount of computing resources. The above-mentioned model instance can represent the above-mentioned running copy, that is, the above-mentioned model instance can represent multiple models as running copies.

[0045] In an optional embodiment, a memory can be pre-established to store the historical data of the above-mentioned large model in history. When scheduling resources for the large model, an acquisition instruction can be sent to the memory to obtain the historical data of the large model corresponding to multiple computing nodes stored in the above-mentioned memory. The user can determine the above-mentioned historical time period according to the needs, obtain the load historical data of multiple computing nodes in the historical time period from the historical data of the multiple computing nodes, and obtain the current operating resources of the large model through sensors. Model instances of the large model are deployed in multiple large models, and the model instances represent the running copies of the large model.

[0046] In another optional embodiment, a cloud can be pre-established, and the historical data of the above-mentioned large model in history can be transmitted to the above-mentioned cloud and stored. When scheduling resources for the large model, the historical data of the large model corresponding to multiple computing nodes in history can be obtained by requesting the cloud to obtain the historical data of the large model corresponding to multiple computing nodes stored in the cloud. The user can determine the above-mentioned historical time period according to the state of the large model, obtain the historical load data of multiple computing nodes in the historical time period from the historical data in the history of multiple computing nodes, and obtain the current operating resources of the large model through sensors.

[0047] In another optional embodiment, a cloud can be pre-established, and the historical data of the large model can be transmitted to the cloud and stored. When scheduling resources for the large model, the historical time period is pre-determined, and the historical time period is sent to the cloud, so as to request the cloud to obtain the historical load data of multiple computing nodes corresponding to the large model, so as to obtain the historical load data of multiple computing nodes in the historical time period, and obtain the current operating resources of the large model through sensors.

[0048] In this application, by obtaining the load historical data of the historical time period and the current running resources of the large model, the resources of the large model are scheduled according to the load historical data and the current running resources, which effectively improves the rationality of scheduling the resources of the large model, effectively improves the accuracy of the large model resource scheduling, and improves the user experience.

[0049] Step S104, inputting the load history data into the load prediction model, using the load prediction model to predict the loads of multiple computing nodes in a preset time period, and obtaining load prediction data.

[0050] The above-mentioned load prediction model refers to a pre-established model for predicting the load of a large model. The above-mentioned load prediction model is used to determine the load of multiple computing nodes of the above-mentioned large model within a preset time period, so as to obtain load prediction data of multiple computing nodes within the preset time period, so as to predict the load of multiple computing nodes of the above-mentioned large model, and then schedule the resources of the above-mentioned large model.

[0051] The above-mentioned load prediction data refers to the load data of multiple computing nodes predicted by the above-mentioned large model within a preset time period. The above-mentioned load prediction data can be used to predict the load consumption of multiple computing nodes of the above-mentioned large model within the preset time period, and then schedule the resources of the above-mentioned large model.

[0052] The above-mentioned preset time period refers to a pre-set time period for predicting the load of multiple computing nodes of the above-mentioned large model, wherein the above-mentioned preset time period can be set according to the state of the large model, the above-mentioned preset time period can also be set manually according to experience and needs, and the above-mentioned preset time period can also be set according to actual application scenarios.

[0053] In an optional embodiment, a load prediction model is established in advance, and a preset time period is determined in advance. After the load history data is obtained, the load history data and the preset time period are used as inputs of the load prediction model. In the load prediction model, the loads of the multiple computing nodes within the preset time period are predicted to obtain the load prediction data and output the load prediction model.

[0054] In this application, the load prediction model is used to accurately predict the load of multiple computing nodes in a preset time period to obtain more accurate load prediction data, thereby improving the accuracy of scheduling resources for large models, effectively improving the efficiency of scheduling resources for large models, and improving the user experience.

[0055] Step S106, determining deployment results of the model instances deployed in the plurality of computing nodes based on the load prediction data, wherein the deployment results are used to indicate whether to adjust the model instances.

[0056] The above deployment result refers to the result of adjusting the model instances deployed in multiple computing nodes in the above large model. The above deployment result can indicate whether the model instance is adjusted. The above deployment result can be to adjust the model instance, or to prohibit the adjustment of the model instance.

[0057] In an optional embodiment, after obtaining the load prediction data, the load prediction data can be compared with a preset threshold to determine the deployment results of the model instances deployed in multiple computing nodes to determine whether the deployment result is to adjust the model instance or to prohibit adjustment of the model instance.

[0058] In another optional embodiment, a deployment result acquisition model may be pre-established, and the load prediction data may be used as input to the deployment result acquisition model to output a deployment result that adjusts the model instance or prohibits adjusting the model instance.

[0059] In this application, the deployment result of whether to adjust the model instance is determined by the above-mentioned load prediction data, which effectively improves the rate of obtaining deployment results, thereby improving the efficiency of scheduling resources in large models, improving the resource utilization of large models, and improving the user experience.

[0060] Step S108: performing resource scheduling based on the deployment result and the currently running resources.

[0061] In an optional embodiment, after obtaining the above deployment results, the user can adjust the model instance, or prohibit the adjustment of the model instance according to the deployment results, and manually schedule resources for multiple computing nodes of the large model in combination with the current running resources.

[0062] In another optional embodiment, after obtaining the above deployment results, a scheduling module can also be pre-established. The above scheduling module adjusts the model instance according to the deployment results, or prohibits the adjustment of the model instance, and performs resource scheduling for multiple computing nodes of the large model in combination with the current running resources.

[0063] In this application, the resources of the large model are quickly and accurately scheduled through deployment results and current running resources, which effectively improves the efficiency and accuracy of scheduling the resources of the large model, thereby effectively improving the utilization rate of the large model resources and improving the user experience.

[0064] Through the above steps, the load history data of multiple computing nodes corresponding to the large model in the historical time period is obtained, and the current running resources of the large model are determined; the load history data is input into the load prediction model, and the load prediction model is used to predict the load of multiple computing nodes in the preset time period to obtain load prediction data; the deployment results of the model instances deployed in the multiple computing nodes are determined based on the load prediction data; and resources are scheduled based on the deployment results and the current running resources. In the present application, the load prediction model is set to predict the load of multiple computing nodes of the large model in the preset time period, and the deployment results of whether to adjust the model instances in the multiple computing nodes of the large model are determined according to the predicted load prediction data, and the resources of the large model are accurately scheduled according to the deployment results and the current running resources, so as to achieve the purpose of effectively improving the accuracy of the resource scheduling of the large model, thereby achieving the technical effect of effectively improving the resource utilization of the large model, and then solving the technical problem of low resource utilization of the large model caused by the low accuracy of resource scheduling of the large model in the related technology.

[0065] Optionally, the deployment results of the model instances deployed in multiple computing nodes are determined based on the load prediction data, including: in response to the load prediction data being greater than a first preset threshold, or, in response to the load prediction data being less than a second preset threshold, determining that the deployment result is to adjust the model instance; in response to the load prediction data being less than or equal to the first preset threshold, and the load prediction data being greater than or equal to the second preset threshold, determining that the deployment result is to prohibit adjustment of the model instance.

[0066] The first preset threshold refers to a preset maximum threshold for determining the adjustment of the model instance. When the load prediction data is greater than the first preset threshold, it is determined to adjust the model instance. The first preset threshold can be set according to the state of the model instance, the first preset threshold can also be set manually according to experience and demand, and the first preset threshold can also be set according to the actual application scenario.

[0067] The second preset threshold refers to a preset minimum threshold for determining the adjustment of the model instance. When the load prediction data is less than the second preset threshold, it is determined to adjust the model instance. The second preset threshold can be set according to the state of the model instance, the second preset threshold can also be set manually according to experience and demand, and the second preset threshold can also be set according to the actual application scenario.

[0068] In an optional embodiment, a first preset threshold and a second preset threshold can be determined according to the state of the above-mentioned model instance. When the load prediction data is greater than the above-mentioned first preset threshold, or the load prediction data is less than the above-mentioned second preset threshold, the deployment result is determined to be to adjust the model instance. When the load prediction data is less than or equal to the first preset threshold, and the load prediction data is greater than or equal to the second preset threshold, the deployment result is determined to prohibit adjustment of the model instance.

[0069] In the present application, the load prediction data is compared with the first preset threshold and the second preset threshold through the first preset threshold to quickly obtain the deployment result of the model instance, thereby effectively improving the rate of scheduling resources for large models, improving the efficiency of scheduling resources for large models, and improving the user experience.

[0070] Optionally, resources are scheduled for the large model based on the deployment results and the current running resources, including: in response to the deployment result being to adjust the model instance, the current running resources are adjusted based on the load forecast data and the load history data to obtain the target running resources, and resources are scheduled according to the target running resources; in response to the deployment result being to prohibit adjustment of the model instance, resources are scheduled based on the current running resources.

[0071] The above-mentioned target operating resources refer to the operating resources of the large model after adjusting the current operating resources. The above-mentioned target operating resources refer to the target operating resources of the above-mentioned large model within the prediction time period obtained according to the above-mentioned deployment results. The resources of the large model can be scheduled according to the above-mentioned target operating resources, which avoids the phenomenon of resource waste and improves the accuracy of scheduling the above-mentioned large model.

[0072] In an optional embodiment, when the deployment result is to adjust the model instance, the user can compare the load forecast data and the load history data to determine the target operating resources of the above-mentioned large model, and schedule the resources of the above-mentioned large model according to the above-mentioned target operating resources; when the deployment result is to prohibit adjustment of the model instance, the resources of the large model are scheduled according to the current operating resources.

[0073] In another optional embodiment, a target operating resource acquisition model can be established in advance. When the deployment result is to adjust the model instance, the above-mentioned load prediction data and load history data are used as inputs of the above-mentioned target operating resource acquisition model, input into the above-mentioned target operating resource acquisition model, output the target operating resources, and schedule the resources of the above-mentioned large model according to the target operating resources; when the deployment result is to prohibit the adjustment of the model instance, the resources of the large model are scheduled according to the current operating resources.

[0074] In this application, load prediction data and load history data are used to accurately determine the target operating resources for resource adjustment of the large model, which effectively improves the accuracy of resource scheduling for the large model, improves the resource utilization of the large model, and improves the user experience.

[0075] Optionally, the current operating resources are adjusted based on the load prediction data and the load history data to obtain the target operating resources, including: in response to the load prediction data being greater than the load history data, determining a first number of model instances to be added, retrieving first operating resources corresponding to the first number from the resource pool, and determining the target operating resources based on the sum of the current operating resources and the first operating resources; in response to the load prediction data being less than the load history data, determining a second number of model instances to be reduced, returning second operating resources corresponding to the second number to the resource pool, and determining the target operating resources based on the difference between the current operating resources and the second operating resources.

[0076] The above-mentioned first number refers to the number of model instances to be added. When the load prediction data is greater than the load historical data, the model instances need to be added. Therefore, it is necessary to determine the first number of model instances to be added. The above-mentioned first number can be set according to the status of the model instance. The above-mentioned first number can also be set manually according to experience and needs. The above-mentioned first number can also be set according to the actual application scenario.

[0077] The above-mentioned second number refers to the number of model instances to be reduced. When the load prediction data is less than the load historical data, the model instances need to be reduced. Therefore, it is necessary to determine the second number of model instances to be reduced. The above-mentioned second number can be set according to the state of the model instance. The above-mentioned second number can also be set manually according to experience and needs. The above-mentioned second number can also be set according to the actual application scenario.

[0078] The above-mentioned resource pool refers to the resource pool for the operation of the above-mentioned large model. The above-mentioned resource pool contains all resources to maintain the operation of the above-mentioned large model. When a model instance needs to be added to the large model, it is scheduled from the above-mentioned resource pool. When the model instance needs to be reduced in the large model, the reduced resources will be returned to the above-mentioned resource pool. In the present application, resource scheduling for the above-mentioned large model is to schedule resources for the large model according to the resources in the resource pool.

[0079] In an optional embodiment, when the load prediction data is greater than the load historical data, the user can manually determine the first number of model instances to be added according to demand, and retrieve the first operating resources corresponding to the first number from the resource pool, and sum the current operating resources and the first operating resources to determine the target operating resources; when the load prediction data is less than the above-mentioned load historical data, the user can manually determine the second number of model instances to be reduced according to demand, and return the second operating resources corresponding to the second number to the resource pool, and subtract the current operating resources from the second operating resources to obtain the difference, and determine the target operating resources based on the difference.

[0080] In another optional embodiment, a quantity determination model may be established in advance, and the load prediction data and load history data may be used as inputs of the quantity determination model and input into the quantity determination model to obtain a first quantity or a second quantity. When the output of the quantity determination model is the first quantity, the first operating resource corresponding to the first quantity is retrieved from the resource pool, and the current operating resource and the first operating resource are summed to determine the target operating resource; when the output of the quantity determination model is the second quantity, the second operating resource corresponding to the second quantity is returned to the resource pool, and the current operating resource and the second operating resource are subtracted to obtain the difference, and the target operating resource is determined based on the difference.

[0081] In the present application, by comparing the load prediction data and the load history data to determine the first quantity or the second quantity, and quickly acquiring the target operating resources based on the first quantity or the second quantity, the efficiency of acquiring the target operating resources is effectively improved, and the user experience is improved.

[0082] Optionally, resource scheduling is performed according to the target operating resources, including: at a preset time point before a preset time period, allocating the target operating resources to the model instance.

[0083] The above-mentioned preset time point refers to a preset time point before the preset time period. Since the large model needs to be scheduled according to the target resources within the preset time period, the target operating resources need to be allocated to the model instance before the preset time period. The above-mentioned preset time point can be set according to the status of the preset time period. The above-mentioned preset time point can also be set manually according to experience and needs. The above-mentioned preset time point can also be set according to the actual application scenario.

[0084] In an optional embodiment, a preset time point is predetermined between preset time periods, and the target operating resources are allocated to the model instance at the preset time point.

[0085] In this application, by setting a preset time point to prevent the target operating resources from being allocated to the model instance within the preset time period, the stability of large model resource scheduling is effectively improved, the resource utilization of the large model is improved, and the user experience is improved.

[0086] Optionally, the method also includes: obtaining sample load history data for a sample historical time period and sample load actual data for a sample preset time period; inputting the sample load history data into an initial load prediction model, and using the initial load prediction model to predict sample load data for multiple computing nodes in the sample preset time period; training the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0087] The above sample historical time period refers to a historical time period used as a sample, and the above sample historical time period can be determined according to user needs.

[0088] The above sample load history data refers to the load history data of the large model within the sample history time period, and the above sample history data can be retrieved from the storage or server.

[0089] The sample preset time period refers to a preset time period used as a sample, and the sample preset time period can be determined according to user needs.

[0090] The above-mentioned sample load actual data refers to the actual data of the sample load in the preset time period of the sample, and the above-mentioned sample load actual data can be retrieved from the storage or the server.

[0091] The above-mentioned initial load prediction model refers to a newly established load prediction model. The initial load prediction model needs to be trained to obtain the final load prediction model. Compared with the above-mentioned load prediction model, the prediction accuracy of the above-mentioned initial load prediction model is lower.

[0092] In an optional embodiment, a user can obtain sample load history data of a sample historical time period and sample load actual data of a sample preset time period from a memory, input the sample load history data into a pre-established initial load prediction model, use the initial load prediction model to predict the sample load data of multiple computing nodes in the sample preset time period, and train the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0093] In this application, by training the initial load prediction model to obtain a more accurate load prediction model, the accuracy of the load prediction model is effectively improved, thereby improving the accuracy of resource scheduling of the large model, improving the resource utilization of the large model, and improving the user experience.

[0094] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0095] Example 2

[0096] The embodiment of the present application also provides a large model resource scheduling device. It should be noted that the large model resource scheduling device of the embodiment of the present application can be used to execute the resource scheduling method for the large model provided in the embodiment of the present application. The resource scheduling device of the large model provided in the embodiment of the present application is introduced below.

[0097] According to an embodiment of the present application, a device for implementing the resource scheduling method of the large model is also provided. Figure 3 is a schematic diagram of the structure of an optional large-scale resource scheduling device according to an embodiment of the present invention, such as Figure 3 As shown, the device includes: an acquisition module 20, which is used to obtain the load history data of multiple computing nodes corresponding to the large model in the historical time period, and determine the current running resources of the large model, wherein the model instances of the large model are deployed in the multiple computing nodes, and the model instances are used to represent the running copies of the large model; a prediction module 22, which is used to input the load history data into the load prediction model, and use the load prediction model to predict the load of multiple computing nodes in a preset time period to obtain load prediction data; a deployment module 24, which is used to determine the deployment results of the model instances deployed in the multiple computing nodes based on the load prediction data, wherein the deployment results are used to indicate whether to adjust the model instances; a scheduling module 26, which is used to perform resource scheduling based on the deployment results and the current running resources.

[0098] Optionally, the deployment module includes: a first judgment unit, used to determine that the deployment result is to adjust the model instance in response to the load prediction data being greater than a first preset threshold, or, in response to the load prediction data being less than a second preset threshold, a second judgment unit, used to determine that the deployment result is to prohibit adjustment of the model instance in response to the load prediction data being less than or equal to the first preset threshold, and the load prediction data being greater than or equal to the second preset threshold.

[0099] Optionally, the scheduling module includes: a first adjustment unit, used to adjust the current operating resources based on load forecast data and load history data in response to the deployment result of adjusting the model instance, obtain the target operating resources, and perform resource scheduling based on the target operating resources; a second adjustment unit, used to schedule resources based on the current operating resources in response to the deployment result of prohibiting adjustment of the model instance.

[0100] Optionally, the second adjustment unit includes: a first judgment subunit, used to determine a first number of model instances to be added in response to the load prediction data being greater than the load historical data, retrieve first operating resources corresponding to the first number from the resource pool, and determine the target operating resources based on the sum of current operating resources and the first operating resources; a second judgment subunit, used to determine a second number of model instances to be reduced in response to the load prediction data being less than the load historical data, return second operating resources corresponding to the second number to the resource pool, and determine the target operating resources based on the difference between the current operating resources and the second operating resources.

[0101] Optionally, the scheduling module further includes: an allocation unit, configured to allocate the target operating resources to the model instance at a preset time point before a preset time period.

[0102] Optionally, the device also includes: an actual acquisition module, used to obtain sample load history data of a sample historical time period and sample load actual data of a sample preset time period; an input module, used to input the sample load history data into an initial load prediction model, and use the initial load prediction model to predict the sample load data of multiple computing nodes in the sample preset time period; a training module, used to train the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0103] Combine the following Figure 4 An optional embodiment of the present invention is described in detail, wherein: Figure 4 is a schematic diagram of the structure of another optional large-model resource scheduling device according to an embodiment of the present invention, such as Figure 4As shown, the device includes: a resource management module 30, a monitoring and data collection module 31, a resource allocation module 32, a load and prediction analysis module 33, an increase model instance 34, a reduction model instance 35, a task issuing module 36, and an advance addition of a model instance 37. After the resource management module 30 receives the large model resource scheduling request sent by the user, the control monitoring and data collection module 31 sends the basic rule adjustment signal to the resource allocation module 32, and sends the load prediction adjustment signal to the load and prediction analysis module 33. The resource allocation module 32 determines the relationship between the load prediction data and the first preset threshold and the second preset threshold according to the basic rule adjustment signal. When the load prediction data is greater than the first preset threshold, or the load prediction data is less than the second preset threshold, it determines to adjust the model instance, and determines to increase the model instance 34 or reduce the model instance 35, and executes it through the task issuing module 36; the load and prediction analysis module 33 predicts the load in the preset time period, obtains the load prediction data, and determines to add the model instance 37 in advance, and also sends the added model instance 37 in advance to the task issuing module 36. The task issuing module 36 controls the resource management module according to the increase of model instance 34, the reduction of model instance 35 and the addition of model instance 37 in advance to schedule resources for the large model.

[0104] Example 3

[0105] According to an embodiment of the present application, an embodiment of a model training method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0106] Under the above operating environment, this application provides Figure 5 The training method of the model shown. Figure 5 is a flow chart of an optional model training method according to an embodiment of the present invention, such as Figure 5 As shown, the method comprises the following steps:

[0107] Step S402, obtaining sample load history data of a sample history time period and sample load actual data of a sample preset time period.

[0108] In an optional embodiment, the sample history time period and the sample preset time period are predetermined, and the user can obtain the sample load history data of the sample history time period and the sample load actual data of the sample preset time period from the server or the storage.

[0109] In the present application, by obtaining the sample load history data of the sample history time period and the sample load actual data of the sample preset time period from the server and storage therein, the training rate of the model is effectively improved and the user experience is improved.

[0110] Step S404: input the sample load history data into the initial load prediction model, and use the initial load prediction model to predict the sample load data of multiple computing nodes in the sample preset time period.

[0111] In an optional embodiment, the sample load history data is used as the input of the above-mentioned initial load prediction model, and is input into the initial load prediction model. The initial load prediction model is used to obtain the sample load prediction data of multiple computing nodes in the sample preset time period, that is, the initial load prediction model outputs the sample load prediction data.

[0112] In the present application, by utilizing the initial load model to obtain sample load prediction data of multiple computing nodes in a sample preset time period, the rate of obtaining sample load prediction data is effectively improved, the rate of model training is improved, and the user experience is improved.

[0113] Step S406: training the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0114] In an optional embodiment, the parameters of the initial load prediction model are adjusted according to the actual load data and the sample load prediction data, that is, the initial load prediction model is trained to obtain a more accurate load prediction model.

[0115] In this application, by training the initial load prediction model based on actual load data and sample load prediction data, the accuracy of the load prediction model is effectively improved, the practicality of model training is improved, and the user experience is improved.

[0116] Example 4

[0117] The embodiment of the present application also provides a model training device. It should be noted that the model training device of the embodiment of the present application can be used to execute the model training method provided in the embodiment of the present application. The model training device provided in the embodiment of the present application is introduced below.

[0118] According to an embodiment of the present application, a device for implementing the training method of the above model is also provided. Figure 6 is a schematic diagram of a structure of an optional model training device according to an embodiment of the present invention, such as Figure 6As shown, the device includes: a sample acquisition module 50, which is used to obtain sample load history data of a sample historical time period and sample load actual data of a sample preset time period; a sample prediction module 52, which is used to input the sample load history data into an initial load prediction model, and use the initial load prediction model to obtain sample load prediction data of multiple computing nodes in the sample preset time period; a sample training module 54, which is used to train the initial load prediction model based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

[0119] Example 5

[0120] An embodiment of the present application may provide an electronic device, Figure 7 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 7 As shown, the electronic device may include: one or more ( Figure 7 (only one is shown) processor 702, memory 704, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0121] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 7 The structure of the electronic device is not limited. Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 7 Different configurations are shown.

[0123] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0124] Example 6

[0125] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0126] Example 7

[0127] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0128] Example 8

[0129] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0130] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0131] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0136] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A resource scheduling method for a large model, characterized in that: include: Obtaining historical load data of multiple computing nodes corresponding to the large model in a historical time period, and determining the current running resources of the large model, wherein model instances of the large model are deployed in the multiple computing nodes, and the model instances are used to represent running copies of the large model; Inputting the load history data into a load prediction model, and using the load prediction model to predict the loads of the plurality of computing nodes in a preset time period to obtain load prediction data; Determining, based on the load prediction data, a deployment result of the model instance deployed in the plurality of computing nodes, wherein the deployment result is used to indicate whether to adjust the model instance; Resource scheduling is performed based on the deployment result and the currently running resources.

2. The resource scheduling method for a large model according to claim 1, characterized in that: Determining a deployment result of the model instance deployed in the plurality of computing nodes based on the load prediction data includes: In response to the load prediction data being greater than a first preset threshold, or in response to the load prediction data being less than a second preset threshold, determining that the deployment result is to adjust the model instance; In response to the load prediction data being less than or equal to the first preset threshold, and the load prediction data being greater than or equal to the second preset threshold, determining that the deployment result is prohibiting adjustment of the model instance.

3. The resource scheduling method for a large model according to claim 1, characterized in that: The large model is scheduled for resources based on the deployment result and the current running resources, including: In response to the deployment result, the model instance is adjusted, the current operating resources are adjusted based on the load forecast data and the load history data to obtain target operating resources, and resource scheduling is performed according to the target operating resources; In response to the deployment result that adjustment of the model instance is prohibited, resource scheduling is performed based on the current running resources.

4. The resource scheduling method for a large model according to claim 3, characterized in that: The current operating resources are adjusted based on the load prediction data and the load history data to obtain target operating resources, including: In response to the load forecast data being greater than the load historical data, determining a first quantity of the model instances to be added, retrieving a first operating resource corresponding to the first quantity from a resource pool, and determining the target operating resource based on a sum of the current operating resource and the first operating resource; In response to the load prediction data being less than the load historical data, a second number of the model instances to be reduced is determined, second operating resources corresponding to the second number are returned to the resource pool, and the target operating resources are determined based on the difference between the current operating resources and the second operating resources.

5. The resource scheduling method for a large model according to claim 3, characterized in that: The resource scheduling is performed according to the target operation resources, including: At a preset time point before the preset time period, the target operating resource is allocated to the model instance.

6. The resource scheduling method for a large model according to claim 1, characterized in that: The method further comprises: Obtaining sample load historical data for a sample historical time period and sample load actual data for a sample preset time period; Inputting the sample load history data into the initial load prediction model, and using the initial load prediction model to predict the sample load data of multiple computing nodes in the sample preset time period; The initial load prediction model is trained based on the sample load actual data and the sample load prediction data to obtain the load prediction model.

7. A model training method, characterized in that: include: Obtaining sample load historical data for a sample historical time period and sample load actual data for a sample preset time period; Inputting the sample load history data into an initial load prediction model, and using the initial load prediction model to obtain sample load prediction data of multiple computing nodes in the sample preset time period; The initial load prediction model is trained based on the sample load actual data and the sample load prediction data to obtain a load prediction model.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images