Neural network model fine tuning reasoning system

Abstract

The invention relates to the technical field of deep learning, and discloses a neural network model fine tuning reasoning system, which mainly comprises a basic hardware equipment layer, a cloud storage interaction layer, a Kubernetes distributed training base and a model application service layer. The Kubernetes distributed training base is used for receiving to-be-trained tasks sent by target nodes distributed in different areas, calling a target neural network model used for training the to-be-trained tasks, calling a target training data set matched with the to-be-trained tasks and uniformly managing the basic hardware equipment layer; and the model application layer is used for storing various target neural network models and adjusting target parameters of the target neural network models when the target neural network models are used for training the to-be-trained tasks, so that output results of the target neural network models are optimal. According to the method, the entry threshold of large model reasoning is reduced, the labor cost of training environment construction and debugging is reduced, and data of a specific scene can be quickly put into model training.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a neural network model fine-tuning reasoning system. Background Art

[0002] In recent years, with the rapid development of computing power and technology in the computer industry, the field of artificial intelligence has ushered in a big explosion and has begun to penetrate into all walks of life. Not long ago, large language models such as ChatGPT and LLaMA appeared in the field of natural speech processing, which can answer a variety of questions in different languages. This has made large models quickly sought after and used by major industries.

[0003] Training a large model with sample data requires a lot of computing resources and other resource costs. However, for fine-tuning a large model, the required computing resources will be greatly reduced. Therefore, it is the best practice for each industry to make some fine-tuning based on the open source large model and its own demand data to make the model more suitable for different scenarios. The existing pre-trained open source large models generally adjust the model structure and code, but it is impossible to quickly put the data of specific scenarios into the model training. Summary of the invention

[0004] In view of this, the present invention provides a neural network model fine-tuning reasoning system to solve the problem of being unable to quickly put data from specific scenarios into model training.

[0005] According to a first aspect, an embodiment of the present disclosure provides a neural network model fine-tuning reasoning system, the system comprising:

[0006] The basic hardware device layer is used to provide the underlying basic application devices;

[0007] Cloud storage interaction layer, used to store multiple target training data sets;

[0008] The Kubernetes distributed training base is used to receive training tasks sent by target nodes distributed in different regions, call the target neural network model used to train the training tasks, call the target training data set that matches the training tasks, and uniformly manage the basic hardware device layer;

[0009] The model application layer is used to store multiple target neural network models and adjust the target parameters of the target neural network model when using the target neural network model to train the task to be trained, so as to optimize the output result of the target neural network model.

[0010] The neural network model fine-tuning reasoning system in the disclosed embodiment mainly includes: a basic hardware device layer, a cloud storage interaction layer, a Kubernetes distributed training base and a model application service layer. The Kubernetes distributed training base is used to receive tasks to be trained sent by target nodes distributed in different areas, and call the target neural network model used to train the tasks to be trained, call the target training data set that matches the tasks to be trained, and uniformly manage the basic hardware device layer; the model application layer is used to store a variety of target neural network models, and when using the target neural network model to train the tasks to be trained, adjust the target parameters of the target neural network model to optimize the output results of the target neural network model. The disclosed embodiment not only lowers the entry threshold for large model reasoning, but also reduces the labor cost of building and debugging the training environment, and can quickly put data for specific scenarios into model training.

[0011] In some optional implementations, the basic hardware device layer includes: a GPU graphics card.

[0012] The disclosed embodiment uses basic hardware devices to assist the neural network model in fine-tuning the inference system to execute the task to be trained.

[0013] In some optional implementations, the Kubernetes distributed training base includes: a training operator component for uniformly managing tasks to be trained.

[0014] The disclosed embodiment uses a training operator component to uniformly manage tasks to be trained.

[0015] In some optional implementations, the Kubernetes distributed training base includes: a volcano component for scheduling tasks to be trained.

[0016] The disclosed embodiment uses the volcano component to schedule tasks to be trained.

[0017] In some optional implementations, the cloud storage interaction layer includes:

[0018] The data storage unit is used to receive various data uploaded by users and use them as various target training data sets to complete the tasks to be trained.

[0019] The disclosed embodiment uses a data storage unit to store various data uploaded by users.

[0020] In some optional implementations, the model application layer includes:

[0021] The model download unit is used to download various target neural network models.

[0022] The disclosed embodiment uses a model downloading unit to download multiple target neural network models.

[0023] In some optional implementations, the model application layer includes:

[0024] The model storage unit is used to store each target neural network model to a target location.

[0025] The disclosed embodiment uses a model storage unit to store the target neural network model.

[0026] In some optional implementations, the model application layer includes:

[0027] The data mounting unit is used to mount the task execution code and the target training data set to the target path.

[0028] The disclosed embodiment uses a data mounting unit to mount the task execution code and the target training data set.

[0029] In some optional implementations, the model application layer includes:

[0030] The startup script unit is used to configure the corresponding startup command for each user.

[0031] The disclosed embodiment configures a startup command for each user by starting a script unit.

[0032] In some optional implementations, the model application layer includes:

[0033] The parameter adjustment unit is used to adjust the target parameters of the target neural network model when using the target neural network model to train the task to be trained.

[0034] The disclosed embodiment uses a parameter adjustment unit to adjust target parameters of a target neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 is a structural block diagram of a neural network model fine-tuning reasoning system according to an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of a model fine-tuning parameter configuration interface according to an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a specific structure of a neural network model fine-tuning reasoning system according to an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of the hardware structure of a computer device according to the present invention. DETAILED DESCRIPTION

[0040] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0042] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0043] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] According to an embodiment of the present invention, a neural network model fine-tuning reasoning system is provided in this embodiment. Figure 1 As shown, the system includes: a basic hardware device layer 11, a cloud storage interaction layer 12, a Kubernetes distributed training base 13 and a model application layer 14.

[0046] exist Figure 1 The basic hardware device layer 11 is used to provide the underlying basic application devices. The cloud storage interaction layer 12 is used to store a variety of target training data sets. The Kubernetes distributed training base 13 is used to receive the tasks to be trained sent by the target nodes distributed in different areas, and call the target neural network model used to train the tasks to be trained, call the target training data set that matches the tasks to be trained, and uniformly manage the basic hardware device layer. The model application layer 14 is used to store a variety of target neural network models, and when using the target neural network model to train the tasks to be trained, adjust the target parameters of the target neural network model to optimize the output results of the target neural network model.

[0047] In a specific example, the basic hardware device layer includes: a GPU graphics card.

[0048] The disclosed embodiment uses basic hardware devices to assist the neural network model in fine-tuning the inference system to execute the task to be trained.

[0049] The neural network model fine-tuning reasoning system in the disclosed embodiment mainly includes: a basic hardware device layer, a cloud storage interaction layer and a model application service layer. Among them, the basic hardware device layer includes: hardware devices such as GPU graphics card hardware, which are uniformly managed and interconnected through the Kubernetes distributed training base (container orchestration engine k8s), and a cloud storage interaction layer is built on the basis of k8s to facilitate the storage and sharing of large models, codes, data and environments (mirror warehouses); the upper model application layer fine-tunes the model through the model fine-tuning code adapted and integrated in the cloud storage, and the final training output model is saved to the cloud storage, and the training model is directly read through the adapted reasoning code and the model service is seamlessly deployed, so that users can directly perform model reasoning applications, etc. The system lowers the entry threshold for large model reasoning, liberates the manpower cost of algorithm personnel to build and debug the training environment, and focuses more on model fine-tuning itself and fits the industry application scenario.

[0050] In some optional implementations, the Kubernetes distributed training base in the embodiments of the present disclosure includes: a training operator component for uniformly managing tasks to be trained.

[0051] The disclosed embodiment uses a training operator component to uniformly manage tasks to be trained.

[0052] In some optional implementations, the Kubernetes distributed training base in the embodiments of the present disclosure includes: a volcano component for scheduling tasks to be trained.

[0053] The disclosed embodiment uses the volcano component to schedule tasks to be trained.

[0054] Specifically, some high-performance GPU servers are managed through k8s to form a training resource pool, and custom resources of the training operator component are deployed on the upper layer of k8s for task management and control of training target nodes. The training operator component manages the tasks to be trained in a unified manner and controls the unified start and stop of the tasks to be trained. In addition, a set of volcano components is deployed on this basis to schedule the tasks to be trained. After the controller integrates the tasks to be trained and sends a request to the volcano component, the volcano component schedules and allocates the tasks by comparing the cluster resources and the size of the resources required by the tasks to be trained. When the resources meet the requirements, the tasks will start at the same time to communicate and collaborate with each other.

[0055] In some optional implementations, the cloud storage interaction layer in the embodiments of the present disclosure includes: a data storage unit.

[0056] Specifically, in order to facilitate algorithm personnel and users to better manage and share data, cloud media storage is used as the storage medium for algorithm models, codes and data. The data storage unit is used to receive various data uploaded by users and use them as various target training data sets to complete the tasks to be trained. When training different types of data, the data sets can be seamlessly replaced to train models for different scenarios and effects; after fine-tuning the training, the model output of the training can be directly accessed and viewed through the media storage.

[0057] At the same time, by designing a cloud storage interaction layer, the Kubernetes distributed training base and the upper model application layer are fine-tuned and reasoned on the model, and users can schedule the underlying training and reasoning resources without noticing. In addition, the storage interaction layer uses cloud media storage to better support the storage of major resources at hand, so as to share, upload or download models and data, etc. The disclosed embodiment can also store the currently commonly used large models directly in cloud storage, and convert them into k8s PVC through cloud storage to mount them in the resource pool of each task to be trained, so as to realize the sharing of one model on multiple machines, greatly reducing the disk occupancy of large models on cluster machines. In general, the storage interaction layer provides a user access interface while greatly saving the storage consumption of large model training.

[0058] In some optional embodiments, the model application layer in the embodiments of the present disclosure includes: a model download unit, used to download multiple target neural network models. A model storage unit, used to store each target neural network model to a target location. A data mounting unit, used to mount the task execution code and the target training data set to a target path. A startup script unit, used to configure a corresponding startup command for each user. A parameter adjustment unit, used to adjust the target parameters of the target neural network model when using the target neural network model to train the task to be trained.

[0059] In order to enable users to quickly tune the neural network model and focus on the data itself and the model effect, the disclosed embodiment integrates commonly used and popular large models into the entire system, so that users only need to provide corresponding data, adjust hyperparameters, etc., to start training fine-tuning with one click, and seamlessly perform model inference and view the overall effect of the model after training. Figure 2 As shown in the figure, it is a schematic diagram of the model fine-tuning parameter configuration interface. Figure 2 In the diagram, the parameter adjustment method, hyperparameter configuration, iteration rounds, batch size, learning rate, resource configuration, computing power application, computing power specification, and number of nodes are displayed.

[0060] In the system, for example, to integrate the Bloomz large model into the system, first download and store the model to the corresponding media storage location, and then mount it directly to a specific location for use during initial debugging and development. Similarly, mount the data and code to a specified path for use. Start the task to be trained online, debug and install the dependencies required for training and reasoning in the container, and package the operating environment and push the image repository through docker for repeated calls later.

[0061] Because the mounting path is fixed, in order to prevent conflicts in parameters and data between different users, the disclosed embodiment develops a customized startup command in the startup script. Each startup will pass the set parameters through environment variables, and the original template will be replaced and copied and stored in a temporary directory specific to each task. This solves the conflict of multiple fine-tuning tasks between users.

[0062] After repeatedly debugging the code to run through the process, all the required variables are defined and passed through the front-end and back-end. When the task is started, the independence of each fine-tuning and reasoning task is achieved through reading and replacing the yaml file and shell script. So far, after the bloomz process is completed, the code can be stored in the cloud media storage. After simply modifying the back-end startup command, it can be called and used by each algorithm user.

[0063] In order to connect model fine-tuning and model inference, the transfer of environment variables is set. On the inference page, you can select the model trained by previous model fine-tuning. Selecting different models will automatically read and replace them in the inference code, thereby achieving the effect of inference code reuse. In general, the model application layer integrates and adapts the training and inference codes to achieve multiple runs of one code, and can make customized changes based on the data and parameters of different users.

[0064] Therefore, the neural network model fine-tuning reasoning system in the disclosed embodiment integrates and optimizes the research work of neural network model fine-tuning in a unified manner, integrates the operating environment, fine-tuning and reasoning code of the current mainstream open source large models, and provides cloud data import interface, training code hyperparameter adjustment interface and other interfaces. Users do not need to worry about debugging and setting up the environment, modifying and adapting the code, etc. They only need to simply select custom data and self-selected hyperparameters on the page to directly use the model fine-tuning reasoning function. It simplifies the tedious steps of fine-tuning and reasoning of large models, allowing users to focus more on the fine-tuning model effect itself.

[0065] In addition, the fine-tuning and inference code of the open source large model is integrated into the system, so that users can directly use their own data to train the desired fine-tuning model without having to debug the environment and modify the code themselves. The cloud media storage sharing method is adopted to reduce the storage space occupied by the large model on the one hand; on the other hand, an interface is provided to allow other storage to interact with the system, so that users can seamlessly transfer their own large amounts of data for system training. A hyperparameter interface is provided externally, and tasks are isolated internally through code and scripts, so that users can train and infer independently without being affected by a shared code. The system has high scalability and can continuously update and be compatible with the fine-tuning and inference of large models in the future, attracting more users to experience and use it.

[0066] like Figure 3 As shown in FIG. 1 , it is a schematic diagram of the specific structure of the neural network model fine-tuning reasoning system disclosed in the present invention. Figure 3 This includes unified management of hyperparameter adjustment, model fine-tuning training, model reasoning, application services, models, codes, data, environment, and Kubernetes.

[0067] The neural network model fine-tuning reasoning system in the disclosed embodiment mainly includes: a basic hardware device layer, a cloud storage interaction layer, a Kubernetes distributed training base and a model application service layer. The Kubernetes distributed training base is used to receive tasks to be trained sent by target nodes distributed in different areas, and call the target neural network model used to train the tasks to be trained, call the target training data set that matches the tasks to be trained, and uniformly manage the basic hardware device layer; the model application layer is used to store a variety of target neural network models, and when using the target neural network model to train the tasks to be trained, adjust the target parameters of the target neural network model to optimize the output results of the target neural network model. The disclosed embodiment not only lowers the entry threshold for large model reasoning, but also reduces the labor cost of building and debugging the training environment, and can quickly put data for specific scenarios into model training.

[0068] The neural network model fine-tuning inference system of the disclosed embodiment can be applied in the power terminal fault detection scenario to execute a power terminal fault detection method. For example, the Kubernetes distributed training base receives the tasks to be trained about the power terminal sent by the target power terminal nodes distributed in different areas, and calls the target neural network model used to train the task to be trained, and calls the target training data set (power terminal fault detection data set) that matches the task to be trained, and then adjusts the target parameters of the target neural network model through the model application layer to optimize the output result of the target neural network model. Finally, the power terminal fault detection is realized quickly, and the purpose of reducing the cost of power terminal fault detection is achieved.

[0069] The neural network model fine-tuning reasoning system of the disclosed embodiment can be applied in a text semantic reasoning scenario to perform a text semantic reasoning method. For example, the Kubernetes distributed training base receives tasks to be trained about text semantics sent by user terminal nodes distributed in different areas, and calls the target neural network model used to train the task to be trained, and calls the target training data set (text data set) matched by the task to be trained, and then adjusts the target parameters of the target neural network model through the model application layer to optimize the output result of the target neural network model. Finally, text semantic reasoning is realized quickly, and the purpose of reducing the cost of text semantic reasoning is achieved.

[0070] The neural network model fine-tuning reasoning system of the disclosed embodiment can be applied in an image recognition scenario to perform an image recognition method. For example, the Kubernetes distributed training base receives tasks to be trained about text semantics sent by image detection nodes distributed in different areas, and calls the target neural network model used to train the task to be trained, as well as calls the target training data set (data set detected by the image detection node) that matches the task to be trained, and then adjusts the target parameters of the target neural network model through the model application layer to optimize the output result of the target neural network model. Finally, image recognition is realized quickly, and the purpose of reducing the cost of image recognition is achieved.

[0071] The embodiment of the present invention further provides a computer device having the above-mentioned neural network model fine-tuning reasoning system. The neural network model fine-tuning reasoning system, such as Figure 1 As shown, the system includes: a basic hardware device layer 11, a cloud storage interaction layer 12, a Kubernetes distributed training base 13 and a model application layer 14. The basic hardware device layer 11 is used to provide the underlying basic application devices. The cloud storage interaction layer 12 is used to store multiple target training data sets. The Kubernetes distributed training base 13 is used to receive the tasks to be trained sent by the target nodes distributed in different areas, and call the target neural network model for training the tasks to be trained, call the target training data set matching the tasks to be trained, and uniformly manage the basic hardware device layer. The model application layer 14 is used to store multiple target neural network models, and when using the target neural network model to train the tasks to be trained, adjust the target parameters of the target neural network model to optimize the output results of the target neural network model. Among them, the basic hardware device layer includes: a GPU graphics card. The Kubernetes distributed training base includes: a trainingoperator component for uniformly managing the tasks to be trained. The Kubernetes distributed training base includes: a volcano component for scheduling the tasks to be trained. The cloud storage interaction layer includes: a data storage unit. The model application layer includes: a model download unit for downloading multiple target neural network models. The model storage unit is used to store each target neural network model to the target location. The data mounting unit is used to mount the task execution code and the target training data set to the target path. The startup script unit is used to configure the corresponding startup command for each user. The parameter adjustment unit is used to adjust the target parameters of the target neural network model when using the target neural network model to train the task to be trained.

[0072] The neural network model fine-tuning reasoning system in the disclosed embodiment mainly includes: a basic hardware device layer, a cloud storage interaction layer, a Kubernetes distributed training base and a model application service layer. The Kubernetes distributed training base is used to receive tasks to be trained sent by target nodes distributed in different areas, and call the target neural network model used to train the tasks to be trained, call the target training data set that matches the tasks to be trained, and uniformly manage the basic hardware device layer; the model application layer is used to store a variety of target neural network models, and when using the target neural network model to train the tasks to be trained, adjust the target parameters of the target neural network model to optimize the output results of the target neural network model. The present invention not only lowers the entry threshold for large model reasoning, but also reduces the manpower cost of building and debugging the training environment, and can quickly put data for specific scenarios into model training.

[0073] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0074] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0075] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0076] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0077] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0078] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0079] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0080] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0081] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A neural network model fine-tuning inference system, characterized in that: The system comprises: The basic hardware device layer is used to provide the underlying basic application devices; Cloud storage interaction layer, used to store multiple target training data sets; The Kubernetes distributed training base is used to receive the tasks to be trained sent by the target nodes distributed in different areas, call the target neural network model used to train the tasks to be trained, call the target training data set matching the tasks to be trained, and uniformly manage the basic hardware device layer; The model application layer is used to store multiple target neural network models, and when using the target neural network model to train the task to be trained, adjust the target parameters of the target neural network model to optimize the output result of the target neural network model.

2. The neural network model fine-tuning inference system according to claim 1, characterized in that: The basic hardware device layer includes: GPU graphics card.

3. The neural network model fine-tuning inference system according to claim 1, characterized in that: The Kubernetes distributed training base includes: a training operator component, which is used to uniformly manage the tasks to be trained.

4. The neural network model fine-tuning inference system according to claim 1, characterized in that: The Kubernetes distributed training base includes: a volcano component, which is used to schedule the tasks to be trained.

5. The neural network model fine-tuning inference system according to claim 1, characterized in that: The cloud storage interaction layer includes: The data storage unit is used to receive various data uploaded by users and use them as various target training data sets to complete the tasks to be trained.

6. The neural network model fine-tuning inference system according to claim 1, characterized in that: The model application layer includes: A model downloading unit is used to download the multiple target neural network models.

7. The neural network model fine-tuning inference system according to claim 6, characterized in that: The model application layer includes: The model storage unit is used to store each target neural network model to a target location.

8. The neural network model fine-tuning inference system according to claim 1, characterized in that: The model application layer includes: The data mounting unit is used to mount the task execution code and the target training data set to the target path.

9. The neural network model fine-tuning inference system according to claim 1, characterized in that: The model application layer includes: The startup script unit is used to configure the corresponding startup command for each user.

10. The neural network model fine-tuning inference system according to claim 1, characterized in that: The model application layer includes: A parameter adjustment unit is used to adjust the target parameters of the target neural network model when using the target neural network model to train the task to be trained.

Images