Model verification method, device and electronic equipment

Abstract

The embodiment of the present invention proposes a model verification method, device and electronic device. The method includes: receiving a model verification task for a first model; searching for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task; judging whether the process is found, and if so, scheduling the model verification task for the first model to the process. The embodiment of the present invention improves the utilization rate of graphics processing unit resources.

Description

Technical Field

[0001] The present invention relates to the field of AI (Artificial Intelligence) technology, and in particular to a model verification method, device and electronic equipment. Background Art

[0002] Deep learning is a new research direction in the field of ML (Machine Learning). It is introduced into machine learning to make it closer to its original goal - artificial intelligence.

[0003] The model trained by the deep learning platform undergoes a series of transformation and packaging operations to form the final algorithm package, which can then be applied to inference products. However, before it is actually used in the product, the effect of the model is generally verified. There are two main existing verification solutions:

[0004] Solution 1: Use an independent inference platform for model verification. Predefine the number of algorithms that can be loaded simultaneously on a GPU (Graphics Processing Unit) card used for model verification. The specific operation process is as follows:

[0005] Step 01: Train the model on the model training platform, export the model to the local computer, convert, encapsulate, add components and engines, and finally package it into an algorithm package;

[0006] Step 02: Import the algorithm package into an independent inference platform, apply for a model verification GPU card, and start the algorithm's inference application on the applied GPU card to perform model verification;

[0007] Step 03: When multiple algorithms need to be verified simultaneously, the number of algorithms that can be loaded on the GPU card is verified according to a pre-defined model, giving priority to filling up one GPU card resource.

[0008] Solution 2: Integrate the inference application in the model training platform and predefine the number of algorithms that can be loaded simultaneously on a training GPU card. The specific operation process is as follows:

[0009] Step 01: The model training platform trains the model and directly converts the model on the platform into an algorithm package of GPU application type;

[0010] Step 02: Verify the algorithm package on the model training platform. If there is no GPU card for model verification, the platform will apply for a training GPU card for the model verification task, load the algorithm package to be verified onto the applied training GPU card, and start the inference application for model verification.

[0011] Step 03: When multiple algorithms need to be verified simultaneously, priority is given to occupying one GPU card resource based on the pre-defined number of algorithms that can be loaded on one training GPU card.

[0012] The defects of the above two solutions are as follows:

[0013] 1. The number of algorithms that can be loaded simultaneously on a model verification or training GPU card is predefined. This cannot automatically meet the needs of a new type of GPU card. This preset value needs to be redefined for the new GPU.

[0014] Second, the number of algorithms that can be loaded simultaneously on a model verification or training GPU card is predefined, which cannot achieve the maximum resource utilization rate. When the algorithm size decreases, more algorithms cannot be automatically loaded for model verification, and when the algorithm size increases, the predefined number of algorithms will not be satisfied;

[0015] 3. Using an independent inference platform requires adding additional servers, which increases costs. Summary of the invention

[0016] The embodiments of the present invention provide a model verification method, device and electronic device to improve the utilization rate of GPU resources.

[0017] The technical solution of the embodiment of the present invention is achieved as follows:

[0018] A model validation method, the method comprising:

[0019] receiving a model validation task for a first model;

[0020] Searching for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task;

[0021] It is determined whether the model is found. If found, the model verification task for the first model is scheduled to the process.

[0022] The determination of whether the search is successful further includes:

[0023] If not found, the computing power required to load the first model is calculated, and a graphics processing unit card that meets the following second condition is searched among the graphics processing unit cards that have been applied for model verification: the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model;

[0024] It is determined whether a graphics processing unit card that meets the second condition is found, and if found, the first model is loaded onto the found graphics processing unit card for model verification.

[0025] The step of determining whether a graphics processing unit card that meets the second condition is found further includes:

[0026] If not found, apply for a graphics processing unit card for model verification from the graphics processing unit resource pool;

[0027] It is determined whether the application is successful. If successful, a process is created, and the first model is loaded onto the graphics processing unit card that has been applied for. The process is used to perform model verification on the first model.

[0028] The determination of whether the application is successful further includes:

[0029] If the application is not obtained, then among the graphics processing unit cards that have been applied for model verification, a graphics processing unit card that meets the following third condition is searched: assuming that after one or more processes on the graphics processing unit card are closed, the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model;

[0030] It is determined whether a graphics processing unit card that meets the third condition is found. If found, one or more processes on the found graphics processing unit card are closed, and the first model is loaded onto the graphics processing unit card for model verification.

[0031] After receiving the model verification task for the first model and before searching for a process that meets the following first condition in all current processes, the method further includes:

[0032] Determine whether there is a graphics processing unit card that has been applied for model verification. If not, apply for a graphics processing unit card for model verification from a graphics processing unit resource pool; if so, perform the action of searching for a process that meets the following first condition among all current processes.

[0033] The receiving of the model verification task for the first model further includes:

[0034] A model training request for a first model is received, and a training graphics processing unit card is applied to a graphics processing unit resource pool. After the application is obtained, a model training process is created.

[0035] A model verification device, comprising:

[0036] A model verification task receiving module, used to receive a model verification task for a first model;

[0037] The model verification task processing module is used to search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model and the process can still be scheduled as a task; determine whether the process is found, and if so, schedule the model verification task for the first model to the process.

[0038] The model verification task processing module determines whether the model is found and then further uses:

[0039] If not found, the computing power required to load the first model is calculated, and a graphics processing unit card that meets the following second condition is searched among the graphics processing unit cards that have been applied for model verification: the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; it is determined whether a graphics processing unit card that meets the second condition is found, and if found, the first model is loaded onto the found graphics processing unit card for model verification.

[0040] After the model verification task processing module determines whether a graphics processing unit card that meets the second condition is found, it is further used to:

[0041] If not found, apply for a graphics processing unit card for model verification from the graphics processing unit resource pool; determine whether the application is successful. If successful, create a process and load the first model to the applied graphics processing unit card, and use the process to perform model verification on the first model.

[0042] The model verification task processing module determines whether the application is successful and is further used to:

[0043] If the application is not successful, then among the graphics processing unit cards that have been applied for model verification, search for a graphics processing unit card that meets the following third condition: assuming that after one or more processes on the graphics processing unit card are closed, the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; determine whether a graphics processing unit card that meets the third condition is found, and if found, close the one or more processes on the found graphics processing unit card, and load the first model onto the graphics processing unit card for model verification.

[0044] The model verification task processing module is further used before searching for a process that meets the following first condition in all current processes:

[0045] Determine whether there is a graphics processing unit card that has been applied for model verification. If not, apply for a graphics processing unit card for model verification from a graphics processing unit resource pool; if so, perform the action of searching for a process that meets the following first condition among all current processes.

[0046] An electronic device comprises any one of the model verification devices described above.

[0047] In an embodiment of the present invention, when a model verification task for a first model is received, if there is currently a model verification process for the first model and the process can still be scheduled with tasks, there is no need to apply for a GPU card from the GPU resource pool, and the verification task of the first model can be directly scheduled to the process, thereby maximizing the use of the locally applied GPU resources and improving the utilization rate of GPU resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flow chart of a model verification method provided by an embodiment of the present invention;

[0049] Figure 2 A flow chart of a model verification method provided by another embodiment of the present invention;

[0050] Figure 3 A flow chart of a model verification method provided by another embodiment of the present invention;

[0051] Figure 4 A flow chart of a model verification method provided by another embodiment of the present invention;

[0052] Figure 5 A flow chart of a model verification method provided by another embodiment of the present invention;

[0053] Figure 6 A flow chart of a model verification method provided by another embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of a model verification device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0056] Figure 1 The model verification method flow chart provided in one embodiment of the present invention has the following specific steps:

[0057] Step 101: Receive a model verification task for a first model.

[0058] Step 102: Search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task.

[0059] Step 103: Determine whether the model is found. If found, schedule the model verification task for the first model to the process.

[0060] In the above embodiment, when a model verification task for the first model is received, if there is currently a model verification process for the first model and the process can still be scheduled, there is no need to apply for a GPU card from the GPU resource pool, and the verification task of the first model can be directly scheduled to the process, thereby maximizing the use of the locally applied GPU resources and improving the utilization rate of the GPU resources;

[0061] Moreover, the above embodiment automatically supports different new types of GPU card resources, such as NVIDIA Tesla T4, P40, P100, V100, etc., without manual intervention to adjust the preset value, saving manpower and being more automated;

[0062] Furthermore, the above embodiment supports verification of models of various sizes, is not limited by the size of the model, and is more user-friendly and automated;

[0063] In addition, the inference backend module can be embedded in the model training backend module, saving the cost of separately configuring the inference platform hardware resources.

[0064] Figure 2 A flow chart of a model verification method provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0065] Step 201: Receive a model verification task for a first model.

[0066] Step 202: Search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task.

[0067] Step 203: Determine whether the search result is found. If found, schedule the model verification task for the first model to the process. If not found, calculate the computing power required to load the first model, and search for a GPU card that meets the following second condition among the GPU cards that have been applied for model verification: the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0068] Step 204: Determine whether a GPU card that meets the second condition is found. If found, load the first model onto the found GPU card for model verification.

[0069] In the above embodiment, when a model verification task for the first model is received, if there is no model verification process for the first model at present, or if there is one but the process can no longer be scheduled, then a GPU card is searched for which the idle computing power is not less than the computing power required to load the first model. If found, the first model is loaded onto the found GPU card for model verification. At this time, there is no need to apply for a GPU card from the GPU resource pool, thereby maximizing the use of the locally applied GPU resources and improving the utilization rate of GPU resources.

[0070] Figure 3 A flow chart of a model verification method provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0071] Step 301: Receive a model verification task for a first model.

[0072] Step 302: Search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled with tasks.

[0073] Step 303: Determine whether the search result is found. If found, schedule the model verification task for the first model to the process. If not found, calculate the computing power required to load the first model, and search for a GPU card that meets the following second condition among the GPU cards applied for model verification: the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0074] Step 304: Determine whether a GPU card that meets the second condition is found. If found, load the first model onto the found GPU card for model verification; if not found, apply to the GPU resource pool for a GPU card for model verification.

[0075] Step 305: Determine whether the application is successful. If successful, create a process and load the first model onto the GPU card that has been applied for. Use the process to perform model verification on the first model.

[0076] In the above embodiment, when a model verification task for the first model is received, after determining that: there is currently no model verification process for the first model, or although it exists, the process can no longer be scheduled for tasks, and the idle computing power of all GPU cards that have been applied for model verification is less than the computing power required to load the first model, a GPU card is applied for from the GPU resource pool, thereby ensuring maximum utilization of the locally applied GPU resources.

[0077] Figure 4 A flow chart of a model verification method provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0078] Step 401: Receive a model verification task for a first model.

[0079] Step 402: Search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task.

[0080] Step 403: Determine whether the search result is found. If found, schedule the model verification task for the first model to the process. If not found, calculate the computing power required to load the first model, and search for a GPU card that meets the following second condition among the GPU cards that have been applied for model verification: the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0081] Step 404: Determine whether a GPU card that meets the second condition is found. If found, load the first model onto the found GPU card for model verification. If not found, apply to the GPU resource pool for a GPU card for model verification.

[0082] Step 405: Determine whether the application is successful. If successful, create a process and load the first model onto the applied GPU card, and use the process to perform model verification on the first model. If not successful, search for a GPU card that meets the following third condition among the GPU cards that have been applied for model verification: assuming that after one or more processes with empty task lists on the GPU card are closed, the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0083] Step 406: Determine whether a GPU card that meets the third condition is found. If found, close the one or more processes with empty task lists on the found GPU card, and load the first model to the GPU card for model verification.

[0084] In the above embodiment, when a model verification task for the first model is received, it is determined that: there is currently no model verification process for the first model, or although it exists, the process can no longer be scheduled with tasks, and the idle computing power of all GPU cards that have been applied for model verification is less than the computing power required to load the first model, and the application for the GPU card from the GPU resource pool fails, then one or more processes with empty task lists in the GPU cards that have been applied for model verification are closed, so that the idle computing power of the GPU card is not less than the computing power required to load the first model, and then the first model is loaded onto the GPU card for model verification, thereby maximizing the utilization of GPU resources without affecting the normal execution of the existing model verification process.

[0085] Figure 5A flow chart of a model verification method provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0086] Step 501: Receive a model verification task for a first model.

[0087] Step 502: Determine whether there is a GPU card that has been applied for model verification. If not, apply for a GPU card for model verification from a GPU resource pool.

[0088] Step 503: If it exists, search among all current processes for a process that meets the following first condition: the process is a model verification process for the first model, and the process can still be scheduled as a task.

[0089] Step 504: Determine whether the model is found. If found, schedule the model verification task for the first model to the process.

[0090] In the above embodiment, when a model verification task for the first model is received, it is first determined whether there is a GPU card that has been applied for model verification. If not, a GPU card is applied for from the GPU resource pool. If there is and there is currently a model verification process for the first model and the process can still be scheduled, there is no need to apply for a GPU card from the GPU resource pool. The verification task of the first model can be directly scheduled to the process, thereby maximizing the use of the locally applied GPU resources and improving the utilization rate of GPU resources.

[0091] Figure 6 A flow chart of a model verification method provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0092] Step 601: The model training client serves as the operation entry. After completing preliminary preparations (such as preparing a training data set), the model training is started and a model training request for the first model is sent to the model training background module. The model training background module receives the model training request and calls the GPU resource pool interface to send a training GPU card application to the GPU resource pool.

[0093] Step 602: The GPU resource pool receives the training GPU card application, allocates a training GPU card, and creates a model training process for the first model; the model training background module receives the first model training completion message reported by the model training process, and puts the first model into the model warehouse of the inference background module.

[0094] After the GPU resource pool receives the application for a training GPU card, it first determines whether it has any remaining training GPU cards. If so, it creates a model training process for the first model.

[0095] Step 603: The model training client receives a model verification request for the first model input by the user, and forwards the request to the model training background module. The model training background module calls the inference background module interface to dispatch a model verification task for the first model to the inference background module.

[0096] Users can access the model warehouse through the model training client and view the various models in the model warehouse. If a model needs to be verified, enter a model verification request for the model on the client.

[0097] Step 604: The inference background module receives the model verification task for the first model, and determines whether it currently has a GPU card for model verification. If so, execute step 606; otherwise, execute step 605.

[0098] Step 605: The inference background module sends a model verification GPU card application to the GPU resource pool. The GPU resource pool receives the application, creates a process, and allocates a GPU card for model verification. The process loads the first model onto the GPU card and starts to verify the first model. This process ends.

[0099] The GPU resource pool will set an upper limit for the allocation of GPU cards for model verification. If the number of GPU cards allocated for model verification exceeds the upper limit, no more GPU cards will be allocated for model verification. In this step, if the number of GPU cards allocated for model verification by the GPU resource pool exceeds the upper limit, the inference background module will be notified, and the inference background module will notify the model training background module, and the model training background module will notify the user of the model verification failure through the model training client and inform the user of the reason.

[0100] After the process is created, it will be registered with the model verification node management module. After that, the model verification node management module can obtain the model verification progress status, GPU card usage status, etc. from the process in real time. The inference background module can then obtain the model verification progress status, GPU card usage status, etc. from the model verification node management module in real time.

[0101] Step 606: The inference background module searches for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task.

[0102] The process can still be scheduled with tasks means that tasks can still be scheduled to the process, for example: the task queue of the process is not full.

[0103] Step 607: The inference background module determines whether the search result is found. If so, execute step 608; otherwise, execute step 609.

[0104] Step 608: The inference background module schedules the model verification task for the first model to the process, and this process ends.

[0105] Step 609: The inference background module calculates the computing power required to load the first model, and searches for a GPU card that meets the following second condition among the GPU cards that have been applied for model verification: the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0106] Step 610: The inference background module determines whether the search result is found. If so, execute step 611; otherwise, execute step 612.

[0107] Step 611: the inference background module specifies a process for the model verification task for the first model. The process loads the first model onto the found GPU card and starts to verify the first model. This process ends.

[0108] Step 612: The inference background module sends a model verification GPU card application to the GPU resource pool. The GPU resource pool receives the application and determines whether the number of GPU cards allocated for model verification has reached a preset upper limit. If so, execute step 614; otherwise, execute step 613.

[0109] The GPU resource pool sets an upper limit for the allocation of GPU cards for model validation. If the number of GPU cards allocated for model validation exceeds this upper limit, no more GPU cards will be allocated for model validation.

[0110] Step 613: The GPU resource pool creates a process and allocates a GPU card for model verification. The process loads the first model onto the GPU card and starts verifying the first model. This process ends.

[0111] Step 614: The GPU resource pool sends an allocation failure notification to the inference background module. The inference background module receives the notification and searches for a GPU card that meets the following third condition among the GPU cards that have been applied for model verification: assuming that after one or more processes with empty task lists on the GPU card are closed, the idle computing power of the GPU card is not less than the computing power required to load the first model.

[0112] Step 615: The inference background module determines whether the model is found. If found, one or more processes with empty task lists of the found GPU card are closed to ensure that the idle computing power of the GPU card is not less than the computing power required to load the first model. The first model is loaded onto the GPU card for model verification.

[0113] In this step, if the inference background module fails to find a GPU card that meets the third condition, the model training background module is notified that the model verification has failed and the reason for the failure is informed. The model training background module then notifies the user through the model training client.

[0114] Figure 7 The schematic diagram of the structure of the model verification device provided by the embodiment of the present invention mainly includes: a model verification task receiving module 71 and a model verification task processing module 72, wherein:

[0115] The model verification task receiving module 71 is used to receive a model verification task for the first model.

[0116] The model verification task processing module 72 is used to search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model and the process can still be scheduled as a task; determine whether the process is found, and if so, schedule the model verification task for the first model to the process.

[0117] In the above embodiment, when a model verification task for the first model is received, if there is currently a model verification process for the first model and the process can still be scheduled with tasks, there is no need to apply for a GPU card from the GPU resource pool, and the verification task of the first model can be directly scheduled to the process, thereby maximizing the use of the locally applied GPU resources and improving the utilization rate of GPU resources.

[0118] In an optional embodiment, the model verification task processing module 72 is further used to determine whether it is found and then: if not found, calculate the computing power required to load the first model, and search for a GPU card that meets the following second condition among the GPU cards that have been applied for model verification: the idle computing power of the GPU card is not less than the computing power required to load the first model; determine whether a GPU card that meets the second condition is found, and if found, load the first model onto the found GPU card for model verification.

[0119] In an optional embodiment, the model verification task processing module 72 determines whether a GPU card that meets the second condition is found and is further used to: if not found, apply for a GPU card for model verification from the GPU resource pool; determine whether the application is successful, and if successful, create a process and load the first model onto the applied GPU card, and use the process to perform model verification on the first model.

[0120] In an optional embodiment, the model verification task processing module 72 is further used to determine whether the application has been successful and: if not, then among the GPU cards that have been applied for model verification, search for a GPU card that meets the following third condition: assuming that after one or more processes with empty task lists on the GPU card are closed, the size of the idle computing power of the GPU card is not less than the computing power required to load the first model; determine whether a GPU card that meets the third condition is found, and if so, close the one or more processes with empty task lists on the found GPU card, and load the first model onto the GPU card for model verification.

[0121] In an optional embodiment, before the model verification task processing module 72 searches for a process that satisfies the following first condition among all current processes, it is further used to: determine whether there is a GPU card that has been applied for model verification, and if not, apply for a GPU card for model verification from the GPU resource pool; if so, execute the action of searching for a process that satisfies the following first condition among all current processes.

[0122] An embodiment of the present invention further provides an electronic device, comprising the model verification device as described above.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A model verification method, characterized in that: Applied to the inference background module, the inference background module is embedded in the model training background module, the model training background module is used to receive the model training request for the first model sent by the model training client after starting the model training, and call the graphics processing unit resource pool interface to send a training graphics processing unit card application to the graphics processing unit resource pool, so that the graphics processing unit resource pool receives the training graphics processing unit card application, allocates the training graphics processing unit card, and creates a model training process for the first model; the model training background module is also used to receive the first model training completion message reported by the model training process, put the first model into the model warehouse of the inference background module, the model training client is used to receive the model verification request for the first model input by the user, and forward the model verification request to the model training background module, and the model training background module is also used to call the inference background module interface to dispatch the model verification task for the first model to the inference background module; The method includes: receiving a model verification task for the first model; Determine whether there is a graphics processing unit card that has been applied for model verification. If not, apply for a graphics processing unit card for model verification from a graphics processing unit resource pool. If it exists, search among all current processes for a process that satisfies the following first condition: the process is a model verification process for the first model, and the process can still be scheduled; Determine whether it is found, and if so, schedule the model verification task for the first model to the process; if not, calculate the computing power required to load the first model, and search for a graphics processing unit card that meets the following second condition among the graphics processing unit cards that have been applied for model verification: the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; Determine whether a graphics processing unit card that meets the second condition is found, and if not found, apply to a graphics processing unit resource pool for a graphics processing unit card for model verification; Determine whether the application is successful. If not, search for a graphics processing unit card that meets the following third condition among the graphics processing unit cards that have been applied for model verification: assuming that after one or more processes with empty task lists on the graphics processing unit card are closed, the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; Determine whether a graphics processing unit card that meets the third condition is found. If found, close the process whose one or more task lists are empty on the found graphics processing unit card, and load the first model onto the graphics processing unit card for model verification; if not found, notify the model training background module that the model verification has failed.

2. The method according to claim 1, characterized in that The step of determining whether a graphics processing unit card satisfying the second condition is found further includes: if found, loading the first model onto the found graphics processing unit card for model verification.

3. The method according to claim 1, characterized in that: The determination of whether the application is successful further includes: If the application is successful, a process is created, and the first model is loaded onto the graphics processing unit card that has been applied for, and the process is used to perform model verification on the first model.

4. The method according to claim 1, characterized in that: The receiving of the model verification task for the first model further includes: A model training request for a first model is received, and a training graphics processing unit card is applied to a graphics processing unit resource pool. After the application is obtained, a model training process is created.

5. A model verification device, characterized in that: Applied to the inference background module, the inference background module is embedded in the model training background module, the model training background module is used to receive the model training request for the first model sent by the model training client after starting the model training, and call the graphics processing unit resource pool interface to send a training graphics processing unit card application to the graphics processing unit resource pool, so that the graphics processing unit resource pool receives the training graphics processing unit card application, allocates the training graphics processing unit card, and creates a model training process for the first model; the model training background module is also used to receive the first model training completion message reported by the model training process, put the first model into the model warehouse of the inference background module, the model training client is used to receive the model verification request for the first model input by the user, and forward the model verification request to the model training background module, and the model training background module is also used to call the inference background module interface to dispatch the model verification task for the first model to the inference background module; The device includes: A model verification task receiving module, used to receive a model verification task for the first model; The model verification task processing module is used to determine whether there is a graphics processing unit card that has been applied for model verification. If not, apply for a graphics processing unit card for model verification from a graphics processing unit resource pool; if so, search for a process that meets the following first condition among all current processes: the process is a model verification process for the first model, and the process can still be scheduled as a task; determine whether it is found, and if so, schedule the model verification task for the first model to the process; if not found, calculate the computing power required to load the first model, and search for a graphics processing unit card that meets the following second condition among the graphics processing unit cards that have been applied for model verification: the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; determine whether a process that meets the second condition is found a graphics processing unit card that meets the second condition; if not found, apply for a graphics processing unit card for model verification from the graphics processing unit resource pool; determine whether the application is successful; if not, search for a graphics processing unit card that meets the third condition among the graphics processing unit cards that have been applied for model verification: assuming that after one or more processes with empty task lists on the graphics processing unit card are closed, the size of the idle computing power of the graphics processing unit card is not less than the computing power required to load the first model; determine whether a graphics processing unit card that meets the third condition is found; if found, close the one or more processes with empty task lists on the found graphics processing unit card, and load the first model onto the graphics processing unit card for model verification; if not found, notify the model training background module that the model verification has failed.

6. The device according to claim 5, characterized in that After the model verification task processing module determines whether a graphics processing unit card that meets the second condition is found, the following further comprises: If found, the first model is loaded onto the found graphics processing unit card for model verification.

7. The device according to claim 5, characterized in that After the model verification task processing module determines whether the application is successful, it further includes: If the application is successful, a process is created, and the first model is loaded onto the graphics processing unit card that has been applied for, and the process is used to perform model verification on the first model.

8. An electronic device, characterized in that: The method comprises a model verification device as claimed in any one of claims 5 to 7.

Images