Model training visualization method and apparatus, and cloud platform

Abstract

The embodiment of the application discloses a model training visualization method and device and a cloud platform. The method comprises the following steps: in response to a model training task request, a training container corresponding to the model training task is created; based on the training container, the model training task is started to be executed, and training logs corresponding to the model training task are collected and stored; in response to a visualization analysis request corresponding to the model training task, the training logs corresponding to the model training task are acquired, and the training logs are input into a visualization analysis container corresponding to the model training task; and the visualization output result of the visualization analysis container is acquired. Through the above method, the model training task can be started to be executed based on the created training container, and the training logs corresponding to the model training task can be collected and stored, so that the visualization analysis container can output the visualization result based on the stored training logs, thereby obtaining the visualization output result corresponding to the model training task without changing the training container.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a model training visualization method, device, and cloud platform. Background Art

[0002] With the advent of the artificial intelligence era, traditional offline deep learning and machine learning model training tasks (such as code writing, model training, hyperparameter debugging, and model deployment) are gradually being replaced by online machine learning platforms. Furthermore, machine learning platforms can visualize model training tasks, allowing developers to adjust them based on the visualizations. However, these visualization methods can disrupt the training environment, preventing the training from proceeding properly. Summary of the Invention

[0003] In view of the above problems, this application proposes a model training visualization method, device and cloud platform to improve the above problems.

[0004] In the first aspect, the present application provides a model training visualization method, which is applied to a cloud platform. The method includes: in response to a model training task request, creating a training container corresponding to the model training task; starting to execute the model training task based on the training container, and collecting and storing the training log corresponding to the model training task; in response to a visualization analysis request corresponding to the model training task, obtaining the training log corresponding to the model training task, and inputting the training log into the visualization analysis container corresponding to the model training task; and obtaining the visualization output result of the visualization analysis container.

[0005] In the second aspect, the present application provides a model training visualization method, which is applied to a user terminal, and the method includes: sending a model training task request to a cloud platform, so that the cloud platform creates a training container corresponding to the model training task, starts executing the model training task based on the training container, and collects and stores the training log corresponding to the model training task; sending a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform obtains the training log corresponding to the model training task, and inputs the training log into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container; and receives the visualization output result returned by the cloud platform.

[0006] In the third aspect, the present application provides a model training visualization device that runs on a cloud platform, and the device includes: a training container creation unit, which is used to create a training container corresponding to the model training task in response to a model training task request; a training log storage unit, which is used to start executing the model training task based on the training container, and collect and store the training log corresponding to the model training task; a training log acquisition unit, which is used to obtain the training log corresponding to the model training task in response to a visualization analysis request corresponding to the model training task, and input the training log into the visualization analysis container corresponding to the model training task; and a visualization result acquisition unit, which is used to obtain the visualization output result of the visualization analysis container.

[0007] In a fourth aspect, the present application provides a model training visualization device that runs on a user terminal, and the device includes: a model training task request sending unit, which is used to send a model training task request to a cloud platform, so that the cloud platform can create a training container corresponding to the model training task, start executing the model training task based on the training container, and collect and store the training log corresponding to the model training task; a visualization analysis request sending unit, which is used to send a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform can obtain the training log corresponding to the model training task, and input the training log into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container; and a result receiving unit, which is used to receive the visualization output result returned by the cloud platform.

[0008] In a fifth aspect, the present application provides a cloud platform comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0009] In a sixth aspect, the present application provides a computer-readable storage medium, in which program code is stored, wherein the above method is executed when the program code is run.

[0010] The present application provides a model training visualization method, device, cloud platform and storage medium. After responding to a model training task request, a training container corresponding to the model training task is created, the model training task is started to be executed based on the training container, and the training log corresponding to the model training task is collected and stored. In response to a visualization analysis request corresponding to the model training task, the training log corresponding to the model training task is obtained, and the training log is input into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container. Through the above method, it is possible to start executing the model training task based on the created training container, and collect and store the training log corresponding to the model training task, so that the visualization analysis container can output the visualization result based on the stored training log, thereby obtaining the visualization output result corresponding to the model training task without changing the training container, so that the developer can adjust the model training task based on the visualization output result, thereby improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of an application environment proposed in this application is shown;

[0013] Figure 2 A flowchart of a model training visualization method proposed in an embodiment of the present application is shown;

[0014] Figure 3 A flowchart of a model training visualization method proposed in another embodiment of the present application is shown;

[0015] Figure 4 A flowchart of a model training visualization method proposed in another embodiment of the present application is shown;

[0016] Figure 5 The following is a structural diagram of a model training visualization device proposed in an embodiment of the present application;

[0017] Figure 6 A structural block diagram of a model training visualization device proposed in another embodiment of the present application is shown;

[0018] Figure 7 Shows a structural block diagram of a cloud platform proposed in this application;

[0019] Figure 8 It is a storage unit for storing or carrying program codes for implementing the model training visualization method according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] With the advent of the artificial intelligence era, model training tasks can be visualized in machine learning platforms. For example, images can be used to represent the changes in the loss function during model training. For another example, images can be used to represent the performance of the model (such as accuracy) under different training parameters.

[0022] During their research, the inventors discovered that certain model training visualization methods can disrupt the environment of the model training task, preventing it from proceeding normally. For example, when a task is interrupted during execution, the corresponding data remains unprocessed and accumulates in the pipeline structure, causing subsequent model training visualization anomalies in the pipeline.

[0023] Therefore, the inventors proposed a model training visualization method, device, and cloud platform in this application, which, after responding to a model training task request, creates a training container corresponding to the model training task, starts executing the model training task based on the training container, and collects and stores the training log corresponding to the model training task, and responds to a visualization analysis request corresponding to the model training task, obtains the training log corresponding to the model training task, and inputs the training log into the visualization analysis container corresponding to the model training task, and obtains the visualization output result of the visualization analysis container. Through the above method, it is possible to start executing the model training task based on the created training container, and collect and store the training log corresponding to the model training task, so that the visualization analysis container can output the visualization result based on the stored training log, thereby obtaining the visualization output result corresponding to the model training task without changing the training container, so that developers can adjust the model training task based on the visualization output result, thereby improving the efficiency of model training.

[0024] In order to better understand the solutions of the embodiments of the present application, the technical terms used in the embodiments of the present application are explained below.

[0025] Kubernetes (abbreviated as k8s): is a portable, scalable open source platform that can be used to manage containerized workloads and services on multiple hosts in a cloud platform, providing application deployment, planning, updating, maintenance and other services.

[0026] k8s node: represents a physical node in k8s. A physical node can be a physical machine (such as a server).

[0027] Pod (container): is the smallest deployable computing unit that can be created and managed in Kubernetes.

[0028] Container images: These can include system libraries, system tools, and other platform environment libraries. They store the files and environments required for container operation. Users can customize container images to suit their containers.

[0029] TensorBoard: is a visualization tool that can visualize the information in the log files output by the machine learning or deep learning model program so that developers can track model training indicators (for example, loss value and accuracy, visualize model graphs, project embeddings into a lower dimensional space, etc.) and adjust the model based on changes in indicators.

[0030] In order to better understand the solution of the embodiment of the present application, an application environment involved in the embodiment of the present application is first introduced below.

[0031] See also Figure 1 , is a schematic diagram of an application environment of an embodiment of the present application. Figure 1 A model training visualization system 10 is provided, which includes a user terminal 100 and a cloud platform 200 connected to the user terminal 100. The user terminal 100 can be used to remotely log in to the cloud platform 200 and remotely control the cloud platform 200 to control the cloud platform 100 to perform related tasks (such as model training tasks, visualization analysis tasks, etc.). The user terminal 100 can include a PC, a tablet computer, a smart panel, etc., which are not limited here.

[0032] The cloud platform 200 can be used to execute related tasks (such as model training tasks, visual analysis tasks, etc.) in response to the control of the user terminal 100. The cloud platform 200 may include a control module, a cluster module, and a log storage module. The control module can be used to interact with the user terminal 100 and control the cluster module and the log storage module. The cluster module can be used to execute related tasks (such as model training tasks, visual analysis tasks, etc.). The log storage module can be used to store training logs corresponding to model training tasks.

[0033] The control module may include a user management module, a visualization management module, and a container management module. The user management module may be used to verify the remote login behavior of the user terminal 100, interact with the user terminal 100 after the login verification is successful, and control the visualization management module and the container management module; the container management module may be used to provide the container image corresponding to the task and control the cluster module to schedule the task instance; the visualization management module may be used to control the cluster module and obtain the training log from the log storage module.

[0034] The cluster module includes a management node and multiple processing nodes. The management node can assign and schedule tasks to multiple processing nodes. The processing nodes can execute related tasks (such as model training tasks and visual analysis tasks) and store the training logs corresponding to the model training tasks in the log storage module. Each processing node can contain multiple containers and a log collection module. The containers can be training containers or visual analysis containers, etc. The log collection module can be used to obtain the training logs corresponding to the model training tasks. The cluster module can be a Kubernetes cluster module, and a processing node can be a server.

[0035] It should be noted that the cloud platform 200 may be a server cluster, and the control cluster module, log storage module, and cluster module may all be composed of multiple servers.

[0036] The embodiments of this application will be described below with reference to the accompanying drawings.

[0037] See also Figure 2 , this application provides a model training visualization method, applied to a cloud platform, the method comprising:

[0038] S110: In response to the model training task request, create a training container corresponding to the model training task.

[0039] The model training task request may be a request for controlling the cloud platform to provide model training services. The model training task request may include the data processing resource configuration required for the model training task. The data processing resource configuration may include CPU (Central Processing Unit) requirement configuration information and GPU (Graphics Processing Unit) requirement configuration information. The model training task may be a training task for a machine learning, deep learning, or other model.

[0040] As a way, the cloud platform can respond to model training task requests and create training containers corresponding to the model training tasks based on data processing resource configuration.

[0041] Optionally, there may be a control for starting a model training task in the interface of the user terminal. In response to the triggering of the control, the user terminal sends a model training task request to the cloud platform; after responding to the model training task request, the user management module of the cloud platform can obtain the data processing resource configuration of the model training task request terminal, and pass the data processing resource configuration through the container management module, so that the container management module can control the cluster module to create a training container corresponding to the model training task based on the data processing resource configuration.

[0042] Optionally, the model training task request may include target conditions for the start of task execution (for example, start execution one hour later, etc.). After receiving the model training task request, the user management module of the cloud platform can detect in real time whether the target conditions are met. If so, the data processing resource configuration of the model training task request terminal can be obtained, and the data processing resource configuration can be passed through the container management module, so that the container management module can control the cluster module to create a training container corresponding to the model training task based on the data processing resource configuration.

[0043] Optionally, the cloud platform may include a cluster module, which may include a management node and multiple processing nodes. The management node may determine a target node from the multiple processing nodes based on the data processing resource configuration and create a training container in the target node. For example, the data processing resource configuration corresponding to the model training task may be one CPU chip and two GPU chips. The management node may then view the current CPU chip and GPU chip usage of each of the multiple processing nodes, select a node from the multiple processing nodes that has at least one unused CPU chip and two unused GPU chips as the target node, and create a training container corresponding to the model training task in the target node.

[0044] S120: Starting to execute the model training task based on the training container, and collecting and storing the training log corresponding to the model training task.

[0045] As a way, after creating the training container, the user management module can obtain the program corresponding to the model training task and the required container image from the user terminal, and control the container management module to configure the image of the created training container based on the required container image, so that the training container can start executing the model training task based on the program and the configured image.

[0046] As one approach, in step S110 , the cloud platform may create a task identifier (TaskID) of the model training task in response to the model training task request, so that the training log may be obtained based on the task identifier.

[0047] Optionally, when starting a model training task based on a training container, you can first specify an output directory for the training logs corresponding to the model training task in the training container and map this output directory to the corresponding collection directory in the log collection module of the target node, allowing the log collection module to collect the training logs. Both the output directory and the collection directory can correspond to the task identifier of the model training task and are configurable variables.

[0048] For example, the mapping between the output directory and the collection directory can be achieved through the following examples:

[0049]

[0050] VolumesMounts can represent the output directory / home / servie / var / ogs / , and volumes can represent the collection directory / home / service / var / logs / mlp-task-log / task-id.

[0051] Optionally, when the training container starts executing the model training task, the training log can be output to the output directory in real time, so that the log collection module can collect the training log in real time and transmit the collected training log to the log storage module in real time. Then, the log storage module can perform persistent storage on the training log. Persistent storage can refer to storing the training log in a database.

[0052] In an embodiment of the present application, by mapping the output directory to the collection directory, the training logs in the training container can be obtained from the log collection module and stored persistently in the log storage module without intruding into the training container (without any modification to the training container).

[0053] S130: In response to a visualization analysis request corresponding to the model training task, obtain a training log corresponding to the model training task, and input the training log into a visualization analysis container corresponding to the model training task.

[0054] The visualization analysis request may be a request for controlling the cloud platform to provide visualization services, the visualization analysis container may refer to a container in the cluster module for providing visualization services, and the visualization service may be implemented through the TensorBoard component.

[0055] In an embodiment of the present application, the visualization analysis request may include a task identifier for the model training task, thereby enabling the cloud platform to confirm for which model training task visualization services are required. As one approach, after creating the task identifier for the model training task in step S110, the cloud platform may send the task identifier to the user terminal and inform the user terminal that the model training task has begun execution.

[0056] As a method, after responding to the visualization analysis request corresponding to the model training task, the user management module can control the visualization management module to obtain the training log corresponding to the model training task from the log storage module, and input the training log into the visualization analysis container corresponding to the model training task.

[0057] Optionally, when inputting the training log into the visualization analysis container corresponding to the model training task, you can first confirm whether the model training task corresponds to a visualization analysis container. If not, you can create a visualization analysis container and input the training log into the created visualization analysis container.

[0058] Optionally, after inputting the training log into the visualization analysis container corresponding to the model training task, the visualization analysis container can obtain visual output results based on the TensorBoard component and the training log.

[0059] Optionally, a visualization analysis request corresponding to the model training task can be sent to the cloud platform after the model training task starts, so that the user can monitor the progress of the model training task in real time.

[0060] In an embodiment of the present application, by deploying the model training task and the visualization analysis task in different containers, the model training task and the visualization analysis task can be isolated. This can avoid the situation where the program corresponding to the model training task runs abnormally or the training log corresponding to the model training task is damaged due to the configuration and startup of the TensorBoard component in the model training task. In addition, isolating the model training task from the visualization analysis task can also reduce the user's manual configuration of the TensorBoard component, thereby improving model training efficiency.

[0061] S140: Obtaining a visualization output result of the visualization analysis container.

[0062] Among them, the visual output results may include content used to help users intuitively understand information related to the model training task in the form of charts, such as the real-time progress of the model training task, changes in model performance (such as accuracy, etc.), and the impact of different training parameters (such as learning rate, optimizer, number of training times, etc.) on model performance.

[0063] As a way, the visual analysis container can send the visual output results to the visual management module in real time.

[0064] Optionally, when the visualization service is implemented through TensorBoard, the visualization output result can be a URL, and information related to the model training task can be obtained by accessing the URL.

[0065] This embodiment provides a model training visualization method, which, after responding to a model training task request, creates a training container corresponding to the model training task, starts executing the model training task based on the training container, collects and stores the training log corresponding to the model training task, and responds to a visualization analysis request corresponding to the model training task, obtains the training log corresponding to the model training task, and inputs the training log into the visualization analysis container corresponding to the model training task, and obtains the visualization output result of the visualization analysis container. Through the above method, it is possible to start executing the model training task based on the created training container, collect and store the training log corresponding to the model training task, so that the visualization analysis container can output the visualization result based on the stored training log, thereby obtaining the visualization output result corresponding to the model training task without changing the training container, so that developers can adjust the model training task based on the visualization output result, thereby improving the efficiency of model training.

[0066] See also Figure 3 , this application provides a model training visualization method, applied to a cloud platform, the method comprising:

[0067] S210: In response to the model training task request, create a training container corresponding to the model training task.

[0068] S220: Start executing the model training task based on the training container, and collect and store the training log corresponding to the model training task.

[0069] S230: In response to the visualization analysis request corresponding to the model training task, obtain the training log corresponding to the model training task.

[0070] The visualization analysis request corresponding to the model training task may include an identifier of the user terminal. The terminal identifier may be used to determine the user terminal that sends the visualization analysis request. Specifically, the terminal identifier may be a user name for the user terminal to log in to the cloud platform.

[0071] S240: Determine whether the user terminal has a corresponding visual analysis container based on the terminal identifier of the user terminal.

[0072] As one approach, the visualization management module can store information indicating whether a terminal identifier has a corresponding visualization analysis container. After responding to a visualization analysis request corresponding to a model training task, the user management module can control the visualization management module to perform a confirmation operation based on the terminal identifier to determine whether a corresponding visualization analysis container exists for the user terminal.

[0073] Optionally, the visualization management module may store the visualization analysis container information corresponding to the terminal identifier in the form of a list.

[0074] For example, as shown in Table 1, Table 1 may indicate whether a visual analysis container currently corresponds to the user terminal, and if so, whether the visual analysis container is idle. When the number of visual analysis containers is 0, it may indicate that the user terminal has no corresponding visual analysis container. When the visual analysis container identifier is -1, it may indicate that there is no visual analysis container. When the visual analysis container idle value is 1, it may indicate that the visual analysis container is idle; when the visual analysis container idle value is -1, it may indicate that there is no visual analysis container; and when the visual analysis container idle value is 0, it may indicate that the visual analysis container is not idle.

[0075] Table 1

[0076]

[0077] In an embodiment of the present application, by setting a terminal identifier for the user terminal and a task identifier for the model training task, and creating corresponding containers for different tasks, the cloud platform can achieve unified management of multi-user tasks and the resources corresponding to the tasks (such as training data, data processing resource configuration, etc.) based on the container, and when executing multi-user tasks at the same time, achieve data isolation of different users and different tasks (visualization analysis tasks and model training tasks), thereby improving the security and stability of cloud platform task execution.

[0078] S250: If yes, confirm whether the visual analysis container corresponding to the user terminal is idle.

[0079] The idle visualization analysis container may mean that the visualization analysis container is not currently executing a visualization analysis task.

[0080] Exemplarily, the visualization management module may confirm whether the visualization analysis container corresponding to the user terminal is idle based on Table 1.

[0081] Optionally, if not, the visualization management module may create a visualization analysis container and input the training log into the created visualization analysis container.

[0082] S260: If idle, use the visual analysis container corresponding to the user terminal as the visual analysis container corresponding to the model training task, and input the training log into the visual analysis container.

[0083] As a method, if the visualization management module determines that the visualization analysis container corresponding to the user terminal is idle, the visualization analysis container corresponding to the user terminal can be used as the visualization analysis container corresponding to the model training task, and the training log can be input into the visualization analysis container.

[0084] S270: If it is not idle, create the visual analysis container and input the training log into the created visual analysis container.

[0085] As a method, if the visualization management module determines that the visualization analysis container corresponding to the user terminal is not idle, a visualization analysis container may be created, and the training log may be input into the created visualization analysis container.

[0086] S280: Obtaining a visualization output result of the visualization analysis container.

[0087] The present embodiment provides a model training visualization method. Through the above-mentioned method, it is possible to start executing the model training task based on the created training container, and collect and store the training logs corresponding to the model training task, so that the visualization analysis container can output the visualization results based on the stored training logs. Thus, the visualization output results corresponding to the model training task can be obtained without changing the training container, so that developers can adjust the model training task based on the visualization output results, thereby improving the efficiency of model training. In addition, in this embodiment, it is possible to determine whether the user terminal has a corresponding visualization analysis container through the terminal identifier. If so, it is then confirmed whether the visualization analysis container is idle. If it is idle, the visualization analysis container corresponding to the user terminal can be directly used as the visualization analysis container corresponding to the model training task, thereby saving data processing resources.

[0088] See also Figure 4 , the present application provides a model training visualization method, applied to a user terminal, the method comprising:

[0089] S310: Send a model training task request to the cloud platform, so that the cloud platform creates a training container corresponding to the model training task, starts executing the model training task based on the training container, and collects and stores the training log corresponding to the model training task.

[0090] As a method, users can access the corresponding interface of the cloud platform through a user terminal and enter the corresponding account and password to log in to the cloud platform. After logging in, users can set up a model training task in the interface. After the setup is completed, the user terminal sends a model training task request to the cloud platform through the user terminal. The cloud platform will create a training container corresponding to the model training task, start executing the model training task based on the training container, and collect and store the training logs corresponding to the model training task.

[0091] S320: Send a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform can obtain the training log corresponding to the model training task, and input the training log into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container.

[0092] As a method, the user can send a visualization analysis request corresponding to the model training task to the cloud platform through the user terminal, so that the cloud platform can obtain the training log corresponding to the model training task, and input the training log into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container.

[0093] S330: Receive the visualization output result returned by the cloud platform.

[0094] As a method, after receiving the visualization output results returned by the cloud platform, the user terminal can adjust the model training task based on the visualization output results. For example, if the user observes the visualization output results and finds that the loss function of the model training task is oscillating at the current learning rate, the user can appropriately reduce the learning rate and repeat the model training task. For another example, if the user observes the visualization output results and finds that the loss function of the model training task has converged and the model accuracy has reached the expected target, the user can terminate the model training task.

[0095] A model training visualization method provided in this embodiment can, through the above-mentioned method, first send a model training task request to the cloud platform, so that the cloud platform creates a training container corresponding to the model training task, starts executing the model training task based on the created training container, and collects and stores the training logs corresponding to the model training task; then, send a visualization analysis request corresponding to the model training task to the cloud platform, so that the visualization analysis container can output visualization results based on the stored training logs, so that the visualization output results corresponding to the model training task can be obtained without changing the training container, so that developers can adjust the model training task based on the visualization output results, thereby improving the efficiency of model training.

[0096] See also Figure 5The present application provides a model training visualization device 600, which runs on a cloud platform. The device 600 includes:

[0097] A training container creation unit 610 is configured to create a training container corresponding to a model training task in response to a model training task request;

[0098] A training log storage unit 620 is configured to start executing the model training task based on the training container, and to collect and store the training log corresponding to the model training task;

[0099] A training log acquisition unit 630 is configured to, in response to a visualization analysis request corresponding to the model training task, acquire a training log corresponding to the model training task, and input the training log into a visualization analysis container corresponding to the model training task;

[0100] The visualization result acquisition unit 640 is configured to acquire the visualization output result of the visualization analysis container.

[0101] As a method, the model training task request includes the data processing resource configuration required for the model training task. The training container creation unit 610 is specifically used to respond to the model training task request and create a training container corresponding to the model training task based on the data processing resource configuration. The data processing resource configuration includes CPU requirement configuration information and GPU requirement configuration information.

[0102] Optionally, the cloud platform includes a cluster module, which includes a management node and multiple processing nodes. The training container creation unit 610 is specifically used for the management node to determine the target node from the multiple processing nodes based on the data processing resource configuration, and to create the training container in the target node.

[0103] As another approach, the training container creation unit 610 is specifically configured to create a task identifier for the model training task in response to the model training task request, and the training log acquisition unit 630 is specifically configured to acquire the training log based on the task identifier.

[0104] As another method, the training log acquisition unit 630 is specifically used to confirm whether the model training task corresponds to a visualization analysis container, which is used to provide visualization services; if not, create the visualization analysis container and input the training log into the created visualization analysis container.

[0105] Optionally, the visualization analysis request includes a terminal identifier, which is used to determine the user terminal that sends the visualization analysis request. The training log acquisition unit 630 is specifically used to determine whether the user terminal has a corresponding visualization analysis container based on the terminal identifier of the user terminal; if so, confirm whether the visualization analysis container corresponding to the user terminal is idle; if so, use the visualization analysis container corresponding to the user terminal as the visualization analysis container corresponding to the model training task, and input the training log into the visualization analysis container; if not idle, create the visualization analysis container, and input the training log into the created visualization analysis container.

[0106] See also Figure 6 The present application provides a model training visualization device 800, which runs on a user terminal and includes:

[0107] A model training task request sending unit 810 is configured to send a model training task request to a cloud platform, so that the cloud platform can create a training container corresponding to the model training task, start executing the model training task based on the training container, and collect and store training logs corresponding to the model training task.

[0108] A visualization analysis request sending unit 820 is configured to send a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform obtains the training log corresponding to the model training task, inputs the training log into the visualization analysis container corresponding to the model training task, and obtains the visualization output result of the visualization analysis container;

[0109] The result receiving unit 830 is configured to receive the visual output result returned by the cloud platform.

[0110] The following will be combined Figure 7 A cloud platform provided by this application is described.

[0111] See also Figure 7 Based on the above-mentioned model training visualization method and apparatus, embodiments of the present application also provide another cloud platform 100 that can execute the above-mentioned model training visualization method. The cloud platform includes one or more (only one is shown in the figure) processors 102 and a memory 104 that are coupled to each other. The memory 104 stores a program that can execute the content of the above-mentioned embodiments, and the processor 102 can execute the program stored in the memory 104.

[0112] The processor 102 may include one or more processing cores. The processor 102 utilizes various interfaces and circuits to connect various components within the cloud platform. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, as well as accesses data stored in the memory 104, to perform various functions and process data within the cloud platform. Optionally, the processor 102 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 102 and may be implemented separately via a communication chip.

[0113] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the terminal 100 during use (such as a phone book, audio and video data, chat history data), etc.

[0114] Please refer to Figure 8 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 1000 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0115] The computer-readable storage medium 1000 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 1000 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 1010 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 1010 can be compressed, for example, in a suitable form.

[0116] In summary, the present application provides a model training visualization method, device, and cloud platform. After responding to a model training task request, a training container corresponding to the model training task is created, the model training task is started to be executed based on the training container, and the training log corresponding to the model training task is collected and stored. In response to a visualization analysis request corresponding to the model training task, the training log corresponding to the model training task is obtained, and the training log is input into the visualization analysis container corresponding to the model training task to obtain the visualization output result of the visualization analysis container. Through the above method, the model training task can be started to be executed based on the created training container, and the training log corresponding to the model training task can be collected and stored, so that the visualization analysis container can output the visualization result based on the stored training log, thereby obtaining the visualization output result corresponding to the model training task without changing the training container, so that the developer can adjust the model training task based on the visualization output result, thereby improving the efficiency of model training.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training visualization method, characterized in that: Applied to a cloud platform, the method includes: In response to a model training task request, create a training container corresponding to the model training task; Starting to execute the model training task based on the training container, and collecting and storing the training log corresponding to the model training task; In response to a visualization analysis request corresponding to the model training task, obtaining a training log corresponding to the model training task, and inputting the training log into a visualization analysis container corresponding to the model training task, wherein inputting the training log into the visualization analysis container corresponding to the model training task includes: confirming whether the model training task corresponds to a visualization analysis container, the visualization analysis container being used to provide visualization services; if not, creating the visualization analysis container, and inputting the training log into the created visualization analysis container; Obtain the visualization output results of the visualization analysis container, wherein the visualization output results include information related to the model training task, and the information related to the model training task includes the real-time progress of the model training task, changes in model performance, and the impact of different training parameters on model performance.

2. The method according to claim 1, characterized in that The model training task request includes the data processing resource configuration required for the model training task, and the step of creating a training container corresponding to the model training task in response to the model training task request includes: In response to the model training task request, a training container corresponding to the model training task is created based on the data processing resource configuration, where the data processing resource configuration includes CPU requirement configuration information and GPU requirement configuration information.

3. The method according to claim 2, characterized in that The cloud platform includes a cluster module, which includes a management node and multiple processing nodes. The step of creating a training container corresponding to the model training task based on the data processing resource configuration includes: The management node determines a target node from the multiple processing nodes based on the data processing resource configuration, and creates the training container in the target node.

4. The method according to claim 1, wherein The method further comprises: In response to the model training task request, creating a task identifier for the model training task; The obtaining of the training log corresponding to the model training task includes: The training log is obtained based on the task identifier.

5. The method according to claim 1, wherein The visual analysis request includes a terminal identifier, and the terminal identifier is used to determine the user terminal that sends the visual analysis request. The confirming whether the model training task corresponds to a visual analysis container includes: Determining whether the user terminal has a corresponding visual analysis container based on the terminal identifier of the user terminal; If yes, inputting the training log into the visual analysis container includes: Confirm whether the visualization analysis container corresponding to the user terminal is idle; If idle, use the visual analysis container corresponding to the user terminal as the visual analysis container corresponding to the model training task, and input the training log into the visual analysis container; If it is not idle, the visual analysis container is created, and the training log is input into the created visual analysis container.

6. A model training visualization method, characterized in that: Applied to a user terminal, the method includes: Sending a model training task request to the cloud platform, so that the cloud platform creates a training container corresponding to the model training task, starts executing the model training task based on the training container, and collects and stores the training log corresponding to the model training task; Sending a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform obtains the training log corresponding to the model training task, and inputs the training log into the visualization analysis container corresponding to the model training task, and obtains the visualization output result of the visualization analysis container, wherein inputting the training log into the visualization analysis container corresponding to the model training task includes: confirming whether the model training task corresponds to a visualization analysis container, and the visualization analysis container is used to provide visualization services; if not, creating the visualization analysis container, and inputting the training log into the created visualization analysis container; Receive the visualization output result returned by the cloud platform, where the visualization output result includes information related to the model training task, including the real-time progress of the model training task, changes in model performance, and the impact of different training parameters on model performance.

7. A model training visualization device, characterized in that: Running on a cloud platform, the device includes: A training container creation unit, configured to create a training container corresponding to the model training task in response to a model training task request; A training log storage unit, configured to start executing the model training task based on the training container, and to collect and store the training log corresponding to the model training task; a training log acquisition unit, configured to, in response to a visualization analysis request corresponding to the model training task, acquire a training log corresponding to the model training task, and confirm whether the model training task corresponds to a visualization analysis container, the visualization analysis container being used to provide visualization services; if not, create the visualization analysis container, and input the training log into the created visualization analysis container; The visualization result acquisition unit is used to acquire the visualization output result of the visualization analysis container.

8. A model training visualization device, characterized in that: Running on a user terminal, the device includes: A model training task request sending unit is used to send a model training task request to the cloud platform, so that the cloud platform creates a training container corresponding to the model training task, starts executing the model training task based on the training container, and collects and stores the training log corresponding to the model training task; A visualization analysis request sending unit is used to send a visualization analysis request corresponding to the model training task to the cloud platform, so that the cloud platform can obtain the training log corresponding to the model training task and confirm whether the model training task corresponds to a visualization analysis container, and the visualization analysis container is used to provide visualization services; if not, create the visualization analysis container, and input the training log into the created visualization analysis container to obtain the visualization output result of the visualization analysis container, the visualization output result includes model training task related information, and the model training task related information includes the real-time progress of the model training task, the change of model performance, and the impact of different training parameters on model performance; The result receiving unit is used to receive the visual output result returned by the cloud platform.

9. A cloud platform, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the control module, wherein the one or more programs are configured to execute the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 5 is executed.

Images