Model warm start method and device, computer equipment, medium and program product

Abstract

The invention relates to the technical field of machine learning, and discloses a model hot start method and device, computer equipment, a medium and a program product. The method comprises the following steps: starting a target server; a target model runs in the target server; the path file and the model file of the target model are located in the same directory; modifying the path file in response to a front-end model switching instruction; when it is detected that the path file is updated, updating the target model according to the modified path file; and restarting the target server based on the updated target model, so that the updated target model runs in the restarted target server. According to the scheme, when the model switching function is achieved, manual switching is not needed, the server does not need to be stopped, model hot start is achieved, and the scheme is flexible and high in efficiency.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a model hot start method, device, computer equipment, medium and program product. Background Art

[0002] A model inference framework is a tool or library used to perform inference and prediction of machine learning models. When the model inference framework is running, it may be necessary to switch between multiple versions of the model to verify the performance difference between different versions of the model.

[0003] In the related art, when switching models, it depends on reading the new model. For example, when the server is running the model, if you switch to the new model, you need to read the weight of the new model into the memory, then manually switch to the new model, and restart the server to run the new model.

[0004] The above solution requires manual switching when switching models, and frequent stopping and restarting of the server will cause obvious delays and low efficiency. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a model hot start method, device, computer equipment, medium and program product to solve the problem of low efficiency in model switching.

[0006] In a first aspect, the present invention provides a model hot start method, the method comprising:

[0007] Starting a target server; a target model is running in the target server; a path file and a model file of the target model are located in the same directory;

[0008] In response to a model switching instruction from the front end, modify the path file;

[0009] When it is detected that the path file is updated, the target model is updated according to the modified path file;

[0010] The target server is restarted based on the updated target model, so that the updated target model runs in the restarted target server.

[0011] In an optional implementation, the target model includes a selective loading code block; and the step of the restarted target server running the updated target model includes:

[0012] When the selectively loaded code block in the target model is executed, the selectively loaded code block is skipped and the code after the selectively loaded code block is executed.

[0013] In an optional implementation, the step of generating the selectively loaded code block includes:

[0014] Traversing the abstract syntax tree corresponding to the target model, calling a check function to check each node in the abstract syntax tree;

[0015] When a target node in the abstract syntax tree meets a preset check condition, an operation statement contained in the target node is replaced with an empty operation statement to obtain a selectively loaded code block corresponding to the target node.

[0016] In an optional implementation, the calling check function checks each node in the abstract syntax tree, including:

[0017] Calling the check function to check the abstract syntax expression of the target node in the abstract syntax tree;

[0018] When the abstract syntax expression is an if statement, it is determined that the target node meets the preset check condition, and the check function returns True.

[0019] In an optional implementation, the front-end process and the target server process are asynchronous processes.

[0020] In an optional implementation, when the target server is restarted based on the updated target model, the front end performs rendering synchronously.

[0021] In a second aspect, the present invention provides a model hot start device, the device comprising:

[0022] A startup module is used to start a target server; a target model is running in the target server; the path file and the model file of the target model are located in the same directory;

[0023] A modification module, used to modify the path file in response to a model switching instruction of the front end;

[0024] An updating module, configured to update the target model according to the modified path file when detecting that the path file is updated;

[0025] The restart module is used to restart the target server based on the updated target model, so that the updated target model runs in the restarted target server.

[0026] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the model hot start method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0027] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the model hot start method of the first aspect or any corresponding embodiment thereof.

[0028] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the model hot start method of the first aspect or any corresponding embodiment thereof.

[0029] The technical solution provided by the present invention may include the following beneficial effects:

[0030] The model hot start method provided by the present invention first starts the target server to run the target model, and the path file and model file of the target model are located in the same directory. When a model switching instruction sent by the front end to indicate model switching is received, the path file of the target model is modified based on the model switching instruction, and it is automatically detected that the path file of the target model is updated, then the target model is updated according to the modified path file to obtain an updated target model, and then the target server is automatically restarted based on the updated target model, so that the restarted target server runs the updated target model without stopping the server, thereby realizing the hot start of the model. The above scheme automatically updates the target model after detecting that the path file of the target model is updated, and automatically restarts the server after the target model is updated, without the need for manual switching or stopping the server. The scheme is flexible, has low delay, and is highly efficient when performing model switching. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 is a schematic flow chart of a model hot start method according to an embodiment of the present invention;

[0033] Figure 2 is a flow chart of another model hot start method according to an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a loading sequence in the related art is shown;

[0035] Figure 4 is a schematic diagram of a loading sequence according to an embodiment of the present invention;

[0036] Figure 5 is a structural block diagram of a model hot start device according to an embodiment of the present invention;

[0037] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0039] A model inference framework is a tool or library used to perform inference and prediction of machine learning models. When the model inference framework is running, it may be necessary to switch between multiple versions of the model to verify the performance difference between different versions of the model.

[0040] In the related technology, when switching models, it is possible to update the model weights without interrupting the service, such as Kubernetes, AWS ECS, Docker Swarm, etc. However, these model inference frameworks all rely on reading the new model when switching models. For example, when the server is running the model, if you switch to a new model, you need to read the weights of the new model into the memory, manually switch to the new model after reading, and restart the server to run the new model.

[0041] The above solution requires manual switching when switching models, and frequent stopping and restarting of the server will cause obvious delays and low efficiency.

[0042] Therefore, an embodiment of the present invention provides a model hot start method, which automatically updates the target model when it is detected that the path file of the model is updated, so as to improve the model switching efficiency and achieve the effect of model hot start.

[0043] According to an embodiment of the present invention, a model hot start method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] In this embodiment, a model hot start method is provided, which can be used for desktop computers, notebook computers, servers, etc. Figure 1 is a flow chart of a model hot start method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0045] Step S101, start the target server.

[0046] The target server may be a software server, which runs on a physical or virtual server and is a software system for receiving, processing and responding to client requests. A target model runs in the target server, and the target model can be selected according to actual needs and configured in the target server. The path file and the model file of the target model are located in the same directory. The path file of the target model is used to indicate the path or name of the target model. The model file of the target model is the specific program code of the target model. The target server calls the corresponding model file based on the path file to run the target model.

[0047] Step S102, in response to the model switching instruction of the front end, modify the path file.

[0048] The front end is the client, which refers to the part that the user can interact directly with, such as web pages, mobile applications, desktop applications, etc. The front end can receive user input, display data or results, and send requests to the back end through the network. The target server of the present invention runs on the back end, which is the part that processes business logic, data storage and calculation, and is invisible to the user. The back end can receive requests sent by the front end, perform model reasoning and process data, and return processing results to the front end.

[0049] When the model needs to be switched, the user sets the model information to be switched, such as model version, model parameters, etc., through interaction with the front end, and sends a model switching instruction to the back end through the front end. After receiving the model switching instruction sent by the front end, the back end modifies the path file corresponding to the target model according to the model information to be switched contained in the model switching instruction, and obtains an updated path file. The updated path file is used to indicate the path or name of the new target model.

[0050] Step S103: when it is detected that the path file is updated, the target model is updated according to the modified path file.

[0051] When it is detected that the path file is updated, that is, the path or name of the target model is updated, the target model can be updated according to the new path or name indicated by the modified path file to obtain an updated target model.

[0052] Step S104: restart the target server based on the updated target model, so that the updated target model runs in the restarted target server.

[0053] After obtaining the updated target model, the target server will automatically update and restart accordingly and run the updated target model.

[0054] The model hot start method provided in this embodiment first starts the target server to run the target model, and the path file and model file of the target model are located in the same directory. When a model switching instruction sent by the front end to indicate model switching is received, the path file of the target model is modified based on the model switching instruction, and it is automatically detected that the path file of the target model is updated, then the target model is updated according to the modified path file to obtain an updated target model, and then the target server is automatically restarted based on the updated target model, so that the restarted target server runs the updated target model to achieve a hot start of the model. The above scheme automatically updates the target model after detecting that the path file of the target model is updated, and automatically restarts the server after the target model is updated. There is no need for manual switching and no need to stop the server. The scheme is flexible, has low latency, and is highly efficient when switching models.

[0055] In this embodiment, a model hot start method is provided, which can be used for desktop computers, notebook computers, servers, etc. Figure 2 is a flow chart of a model hot start method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0056] Step S201, building a project framework.

[0057] Optionally, the project framework is an asynchronous framework, and the front-end and back-end are written using different processes, so that the front-end process and the target server process are asynchronous processes.

[0058] Exemplarily, a project framework is built based on uvicorn and fastapi. uvicorn is a fast ASGI (Asynchronous Server Gateway Interface) server for building asynchronous Web services, and fastapi is a high-performance Python web framework. Figure 3 A schematic diagram of the loading sequence in the related art is shown. Figure 4 It is a schematic diagram of the loading sequence according to an embodiment of the present invention. The front end is mainly written in Javascript, and the back end is mainly written in Python. In the project framework in the related art, when the back end needs to switch models, it is necessary to shut down the currently running server of the back end, re-read the new model, and then start and load the back end data and front end pages in sequence. However, the present invention uses different processes to write the front end and the back end, and the front end and the back end are started and run separately. When the back end needs to switch models, it can switch and update automatically without affecting the page display of the front end, without shutting down the server, and without loading the back end data and the front end pages in sequence.

[0059] Step S202: Generate a selective loading code block.

[0060] In the process of running the model reasoning framework, the target model will be loaded into the memory and connected to various databases or network links. If the target model is switched during the development process, the target server needs to be restarted, which means that the new target model will be reloaded or the various databases or network links will be reconnected, which will introduce a lot of delays between development cycles. Based on this, the present invention introduces a selective loading code block, and developers can use the selective loading code block to mark the code that does not need to be reloaded. Combined with the front-end and back-end asynchronous project framework, the selective loading code block can be skipped when reloading the new target model without reconnecting to various databases or network links.

[0061] Optionally, the Abstract Syntax Trees (AST) corresponding to the target model is traversed, and a check function is called to check each node in the abstract syntax tree. When the target node in the abstract syntax tree meets the preset check conditions, the operation statement contained in the target node is replaced with an empty operation statement to obtain a selectively loaded code block corresponding to the target node. The preset check conditions can be set according to actual needs, for example, according to the syntax characteristics of the code connected to various databases or network links. Specifically, when checking the target node in the abstract syntax tree, the check function is called to check the abstract syntax expression of the target node in the abstract syntax tree. When the abstract syntax expression is an if statement, it is determined that the target node meets the preset check conditions, and the check function returns True.

[0062] After the selective loading code block is generated, the code of the target model that needs to be loaded is rewritten, and the selective loading code block that does not need to be loaded can be skipped when loading the target model.

[0063] Step S203, start the target server.

[0064] The target server runs a target model. The path file and the model file of the target model are located in the same directory.

[0065] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0066] Step S204, in response to the model switching instruction of the front end, modify the path file.

[0067] For details, please see Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0068] Step S205: when it is detected that the path file is updated, the target model is updated according to the modified path file.

[0069] For details, please see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0070] Step S206: restart the target server based on the updated target model, so that the updated target model runs in the restarted target server.

[0071] When the target model includes a selective loading code block, when the target server is restarted to run the updated target model, when the selective loading code block in the target model is run, the selective loading code block is skipped and the code after the selective loading code block is run.

[0072] When the target server is restarted based on the updated target model, the front-end is rendered synchronously. The restart of the back-end target server will not affect the front-end page display, and the front-end will automatically determine whether rendering is required based on demand.

[0073] The model hot start method provided in this embodiment first starts the target server to run the target model, and the path file and model file of the target model are located in the same directory. When a model switching instruction sent by the front end to indicate model switching is received, the path file of the target model is modified based on the model switching instruction, and it is automatically detected that the path file of the target model is updated, then the target model is updated according to the modified path file to obtain an updated target model, and then the target server is automatically restarted based on the updated target model, so that the restarted target server runs the updated target model to achieve a hot start of the model. The above scheme automatically updates the target model after detecting that the path file of the target model is updated, and automatically restarts the server after the target model is updated. There is no need for manual switching and no need to stop the server. The scheme is flexible, has low latency, and is highly efficient when switching models.

[0074] In the present embodiment, a model hot start device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0075] This embodiment provides a model hot start device, such as Figure 5 As shown, including:

[0076] The startup module 501 is used to start the target server; the target server runs a target model; the path file and the model file of the target model are located in the same directory;

[0077] A modification module 502, for modifying the path file in response to a model switching instruction from the front end;

[0078] An updating module 503 is used to update the target model according to the modified path file when it is detected that the path file is updated;

[0079] The restart module 504 is used to restart the target server based on the updated target model, so that the updated target model runs in the restarted target server.

[0080] In an optional implementation, the target model includes a selective loading code block; the restart module is further used to: when running to the selective loading code block in the target model, skip the selective loading code block and run the code after the selective loading code block.

[0081] In an optional embodiment, the device also includes a selective loading code block module, which is used to: traverse the abstract syntax tree corresponding to the target model, and call a check function to check each node in the abstract syntax tree; when the target node in the abstract syntax tree meets the preset check conditions, replace the operation statement contained in the target node with an empty operation statement to obtain a selective loading code block corresponding to the target node.

[0082] In an optional embodiment, the selectively loaded code block module is also used to: call the check function to check the abstract syntax expression of the target node in the abstract syntax tree; when the abstract syntax expression is an if statement, it is determined that the target node meets the preset check condition, and the check function returns True.

[0083] In an optional implementation, the process of the front end and the process of the target server are asynchronous processes.

[0084] In an optional implementation, when the target server is restarted based on the updated target model, the front end performs rendering synchronously.

[0085] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0086] The model hot start device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0087] The embodiment of the present invention also provides a computer device having the above Figure 5 Model shown is a hot start device.

[0088] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0089] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0090] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0091] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0092] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0093] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0094] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0095] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0096] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0097] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall all fall within the scope of protection of the present invention.

Claims

1. A model hot start method, characterized in that: The method comprises: Starting a target server; a target model is running in the target server; a path file and a model file of the target model are located in the same directory; In response to a model switching instruction from the front end, modify the path file; When it is detected that the path file is updated, the target model is updated according to the modified path file; The target server is restarted based on the updated target model, so that the updated target model runs in the restarted target server.

2. The method according to claim 1, characterized in that: The target model includes a selective loading code block; the step of the restarted target server running the updated target model includes: When the selectively loaded code block in the target model is executed, the selectively loaded code block is skipped and the code after the selectively loaded code block is executed.

3. The method according to claim 2, characterized in that The steps of generating the selective loading code block include: Traversing the abstract syntax tree corresponding to the target model, calling a check function to check each node in the abstract syntax tree; When a target node in the abstract syntax tree meets a preset check condition, an operation statement contained in the target node is replaced with an empty operation statement to obtain a selectively loaded code block corresponding to the target node.

4. The method according to claim 3, characterized in that The calling check function checks each node in the abstract syntax tree, including: Calling the check function to check the abstract syntax expression of the target node in the abstract syntax tree; When the abstract syntax expression is an if statement, it is determined that the target node meets the preset check condition, and the check function returns True.

5. The method according to claim 1, characterized in that The process of the front end and the process of the target server are asynchronous processes.

6. The method according to claim 5, characterized in that When the target server is restarted based on the updated target model, the front end performs rendering synchronously.

7. A model hot start device, characterized in that: The device comprises: A startup module is used to start a target server; a target model is running in the target server; the path file and the model file of the target model are located in the same directory; A modification module, used to modify the path file in response to a model switching instruction of the front end; An updating module, configured to update the target model according to the modified path file when detecting that the path file is updated; The restart module is used to restart the target server based on the updated target model, so that the updated target model runs in the restarted target server.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the model hot start method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the model hot start method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the model hot start method according to any one of claims 1 to 6.

Images