Resource scheduling method and device, storage medium and computer program product

Abstract

The invention discloses a resource scheduling method and device, a storage medium and a computer program product, and relates to the technical field of machine learning, and the method comprises the steps: setting a callback mechanism and a timed task in an editor when the starting state of the editor is not started, a debugging container corresponding to the editor is automatically deleted and stopped through the callback mechanism so as to recover computing power; triggering the debugging container to start a code debugging process through a pre-established communication tunnel; based on a pre-obtained computing power configuration instruction, applying for computing power required by operation of the debugging container; and in the code debugging process, idle computing power detection and recovery are carried out on the computing power required by operation of the debugging container based on the timed task. The computing power of the attached debugging container of the unstarted editor is recovered by setting a callback mechanism, idle computing power detection and recovery are carried out on a timed task, and the resource utilization rate is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a resource scheduling method, device, storage medium, and computer program product. Background Art

[0002] Since the machine learning modeling system completely uses cloud technology to provide code editing modeling services, there is no need to build a local compilation environment. Users can perform machine learning modeling through a browser, and computing resources in a cloud environment are shared and limited. Therefore, model building through a machine learning modeling system requires improving the utilization of computing resources to avoid idle resources. At present, machine learning modeling systems usually place code editors and debugging containers in the same container, resulting in GPU resources being occupied even when there is no debugging running, resulting in a waste of computing resources and failing to make efficient use of limited computing resources.

[0003] Therefore, how to improve the computing power utilization of the editor and debugging container in the machine learning modeling system and avoid resource idling has become an urgent problem to be solved in this application.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a resource scheduling method, device, storage medium and computer program product, aiming to solve the technical problem of how to improve the running computing power utilization of editors and debugging containers in machine learning modeling systems and avoid resource idling.

[0006] To achieve the above objectives, the present application proposes a resource scheduling method, which is applied to a machine learning modeling system, wherein the machine learning modeling system includes an editor and a debugging container, and the method includes:

[0007] When the startup status of the editor is not started, a callback mechanism and a scheduled task are set in the editor, and the debugging container corresponding to the editor is automatically deleted and stopped through the callback mechanism to recycle computing power;

[0008] Triggering the debugging container to start a code debugging process through a pre-established communication tunnel;

[0009] Apply for the computing power required for debugging container operation based on the computing power configuration instructions obtained in advance;

[0010] In the code debugging process, idle computing power detection and recovery are performed on the computing power required for running the debugging container based on the scheduled task.

[0011] In one embodiment, when the startup status of the editor is not started, before the step of setting a callback mechanism and a scheduled task in the editor, and automatically deleting and stopping the debugging container corresponding to the editor through the callback mechanism to recycle computing power, the step further includes:

[0012] An operation of running computing power separation is performed on the editor and the debugging container.

[0013] In one embodiment, when the startup status of the editor is not started, after the step of setting a callback mechanism and a scheduled task in the editor, and automatically deleting and stopping the debugging container corresponding to the editor through the callback mechanism to recycle computing power, the step further includes:

[0014] Starting an editor with a callback mechanism and a scheduled task according to a pre-acquired editor startup instruction;

[0015] Capture interactive signals to identify machine learning modeling code through an editor with callback mechanisms and scheduled tasks set up in the startup state.

[0016] In one embodiment, before the step of triggering the debugging container to start the code debugging process through the pre-established communication tunnel, the step further includes:

[0017] Generate an encrypted public key according to the port forwarding protocol of the editor, and write the encrypted public key into a public configuration directory;

[0018] Read the encryption public key from the public configuration directory to configure the encryption server of the debugging container;

[0019] The debugging container encryption server writes the debugging container IP into a public configuration directory, and establishes a communication tunnel between the editor and the debugging container.

[0020] In one embodiment, the step of triggering the debugging container to start the code debugging process through the pre-established communication tunnel includes:

[0021] Acquire a code debugging instruction issued by a user through the communication tunnel;

[0022] The debugging container is triggered to enter a code debugging process according to the code debugging instruction.

[0023] In one embodiment, the step of applying for computing power required for running the debugging container based on the pre-acquired computing power configuration instruction includes:

[0024] Obtain computing power configuration instructions and monitor whether the user enters computing power adjustment instructions;

[0025] If a computing power adjustment instruction input by the user is detected, the computing power required for the debugging container operation is applied according to the computing power adjustment instruction;

[0026] If the computing power adjustment instruction input by the user is not detected, the computing power required for the debugging container operation is applied for according to the computing power configuration instruction.

[0027] In one embodiment, in the code debugging process, the step of detecting and recovering idle computing power based on the scheduled task for the computing power required for running the debugging container includes:

[0028] In the code debugging process, the computing power required for running the debugging container is detected to be idle through the scheduled task, and the idle time of the debugging container is calculated;

[0029] If the debugging container is idle for a period of time exceeding a preset threshold, the debugging container is automatically deleted and stopped, and the idle computing power required for running the debugging container is recovered.

[0030] In addition, to achieve the above objectives, the present application also proposes a resource scheduling device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the resource scheduling method described above.

[0031] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the resource scheduling method described above are implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the resource scheduling method described above are implemented.

[0033] One or more technical solutions proposed in this application have at least the following technical effects:

[0034] When the editor is not started, a callback mechanism and a scheduled task are set. Through the callback mechanism, it is ensured that when the editor is recycled, its attached debugging container will also be automatically deleted and stopped, thereby avoiding the debugging container from still occupying valuable computing resources when the editor is not in use; the debugging container is triggered to start the code debugging process through a pre-established communication tunnel, so that the debugging container is occupied only when code debugging is required to avoid unnecessary waste of computing power; the required initial computing power configuration is determined according to the machine learning modeling code entering the code debugging process, and based on the initial computing power configuration, the computing power required for the corresponding debugging container operation is applied to achieve a reasonable allocation of computing power resources; in the code debugging process, the running status of the debugging container is monitored through a scheduled task, and idle computing power is discovered in time. Once it is found that the debugging container is idle for more than the set threshold, the debugging container is deleted and stopped and the computing power resources occupied by it are released. In summary, this application reclaims the computing power of the debugging container attached to the editor in an inactive state by setting a callback mechanism, starts the debugging container on demand and allocates reasonable computing power resources, and performs idle computing power detection and recycling in time to avoid long-term idling and waste of resources, effectively improving resource utilization and turnover rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the resource scheduling method of the present application;

[0038] Figure 2 A flowchart of the second embodiment of the resource scheduling method of the present application is provided;

[0039] Figure 3 A logical diagram of computing resource scheduling for starting the editor;

[0040] Figure 4 A logical diagram of computing resource scheduling for starting and debugging containers;

[0041] Figure 5 This is a schematic diagram of the module structure of the resource scheduling device according to an embodiment of the present application;

[0042] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the resource scheduling method in the embodiment of the present application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] The main solution of the embodiment of the present application is: on the basis of separating the computing power of the editor and the debugging operation, a callback mechanism and a scheduled task are set for the editor in the unstarted state; according to the pre-established communication tunnel between the editor and the debugging container, the debugging tool used for code debugging can realize the communication of input, output and control messages across containers, thereby triggering the debugging container to start the code debugging process; based on the pre-acquired computing power configuration instructions, the computing power resources required for the operation of the debugging container are applied for; in the process of code debugging, the computing power required for the operation of the debugging container is detected and recycled through the scheduled task.

[0047] In this embodiment, for ease of description, the following description is made by identifying the machine learning modeling system as the execution entity.

[0048] The embodiments of the present application take into account that: since the machine learning modeling system completely uses cloud technology to provide code editing modeling services, there is no need to build a local compilation environment, and users can perform machine learning modeling through a browser, while computing resources in a cloud environment are shared and limited. Therefore, model building through a machine learning modeling system requires improving the utilization of computing resources to avoid idling resources. At present, machine learning modeling systems usually place code editors and debugging containers in the same container, resulting in GPU resources being occupied even when there is no debugging operation, resulting in a waste of computing resources, and not making efficient use of limited computing resources.

[0049] Therefore, the present application provides a solution. When the editor is not started, a callback mechanism and a scheduled task are set. Through the callback mechanism, it is ensured that when the editor is recycled, its attached debugging container will also be automatically deleted and stopped, thereby avoiding the debugging container from still occupying valuable computing resources when the editor is not in use; triggering the debugging container to start the code debugging process through a pre-established communication tunnel, so that the debugging container is occupied only when code debugging is needed to avoid unnecessary waste of computing power; applying for the computing power required for the operation of the corresponding debugging container based on the computing power configuration instruction, and realizing the reasonable allocation of computing power resources; in the code debugging process, the operation of the debugging container is monitored through a scheduled task, and idle computing power is discovered in time. Once it is found that the debugging container is idle for more than the set threshold, the debugging container is deleted and stopped and the computing power resources occupied by it are released. In summary, the present application recovers the computing power of the debugging container attached to the editor in an inactive state by setting a callback mechanism, starts the debugging container on demand and allocates reasonable computing power resources, and detects and recycles idle computing power in time, avoiding long-term idling and waste of resources, and effectively improving resource utilization and turnover rate.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a machine learning modeling system, etc. The following takes the machine learning modeling system as an example to illustrate this embodiment and the following embodiments.

[0051] Based on this, an embodiment of the present application provides a resource scheduling method, which is applied to a machine learning modeling system, wherein the machine learning modeling system includes an editor and a debugging container. Figure 1 , Figure 1 This is a flowchart of the first embodiment of the resource scheduling method of the present application.

[0052] In this embodiment, the machine learning modeling system includes an editor and a debugging container, the debugging container is separated from the editor computing power, and the resource scheduling method includes steps S10 to S40:

[0053] Step S10, when the startup status of the editor is not started, setting a callback mechanism and a scheduled task in the editor, and automatically deleting and stopping the debugging container corresponding to the editor through the callback mechanism to recycle computing power;

[0054] The editor is served by the IDE host container, and the debugging container provides the debugging and running capabilities of the code. The computing power of the two is separated. Each editor can start a debugging container. Each debugging container has a unique identifier and is attached to the editor.

[0055] The callback mechanism is the Kubernetes (K8s) callback mechanism. The Kubernetes callback mechanism triggers callback functions or hooks in various resources and components when specific events occur, inserting user logic at key points in the resource life cycle to better control various resources and components. By setting the Kubernetes callback mechanism, when the editor is inactive, the computing power resource recovery of the debug container attached to the editor is automatically triggered.

[0056] Scheduled tasks refer to tasks or programs that are automatically executed according to a predetermined time plan. They are used to monitor the idle status of debugging containers and automatically delete the container to reclaim computing resources when it is idle for more than a set period of time.

[0057] In the cloud-based separation architecture, the editor and debugging container are relatively independent entities. When the editor is not started, it means that the user is not currently editing or debugging code. The machine learning modeling system sets a callback function on the editor container to form a callback mechanism. This function is triggered when the editor container is stopped or not started.

[0058] When this callback function is triggered, the machine learning modeling system automatically deletes and stops the debugging container associated with the current editor, releasing the computing resources occupied by the debugging container so that these resources can be used by other users or tasks.

[0059] A background scheduled task is started for each editor, which is responsible for regularly monitoring the idleness of the debugging container. If the debugging container is idle for a set period of time (that is, no debugging tasks are executed), the machine learning modeling system will mark the debugging container as recyclable and delete it at an appropriate time to recycle computing resources.

[0060] Step S20, triggering the debugging container to start the code debugging process through the pre-established communication tunnel;

[0061] In order to achieve seamless communication between the editor and the debugging container, the machine learning modeling system pre-establishes a communication tunnel between the editor and the debugging container through the SSH port forwarding protocol, which is the SSH communication tunnel.

[0062] It should be noted that SSH tunneling is a method of using the SSH protocol to encrypt other protocols or network links. It can forward network data through the SSH connection and automatically provide corresponding encryption and decryption services.

[0063] The user writes machine learning modeling code in the editor and triggers the code debugging function through interface operations. After the editor receives the user's debugging request, it sends the request to the debugging container through the SSH tunnel. After receiving the request, the debugging container parses the request and starts the corresponding code debugging process. The debugging process will run the code written by the user and feed back the debugging results to the editor in real time. During the debugging process, the editor and the debugging container communicate in real time through the SSH tunnel to ensure that the user can view the debugging results and modify the code in real time.

[0064] Step S30, applying for the computing power required for running the debugging container based on the computing power configuration instruction obtained in advance;

[0065] When the user completes writing the machine learning modeling code in the editor and is ready to enter the debugging phase, the machine learning modeling system first needs to confirm an appropriate initial computing power configuration to start the debugging container. This initial computing power configuration is determined based on the machine learning modeling code submitted by the user. The machine learning modeling system analyzes the logic complexity, data volume, model size and other factors in the code through the computing power configuration instructions pre-set by the user, determines the computing power requirements of the debugging container (such as whether a GPU is required, how many GPUs are required, memory requirements, etc.), and comprehensively determines a computing power configuration that meets the debugging requirements without excessively wasting resources.

[0066] The purpose of determining the computing power configuration for debugging container operation is to efficiently utilize computing power resources during the debugging phase, avoiding debugging failures due to insufficient computing power or resource waste due to excess computing power. Therefore, based on the pre-acquired computing power configuration instructions, the computing power required for debugging container operation is applied to achieve on-demand application of computing power resources, avoid wasting computing power resources, and indirectly improve the utilization rate of computing power resources.

[0067] Step S40: During the code debugging process, idle computing power detection and recovery are performed on the computing power required for running the debugging container based on the scheduled task.

[0068] When the user starts debugging the code, the machine learning modeling system will start the corresponding debugging container according to the user's needs and allocate corresponding computing resources (such as GPU). At the same time, the machine learning modeling system will start a background scheduled task for each editor, which is responsible for regularly monitoring the usage of the debugging container.

[0069] By monitoring the code running process in the debugging container, the machine learning modeling system can determine whether the debugging container is idle. If there is no code running process for a period of time, the debugging container is considered to be idle. Through the idle computing power detection and recycling mechanism, the machine learning modeling system can promptly discover and recycle the debugging container that is no longer needed to avoid the waste of computing power resources. This allows scarce resources such as GPUs and memory to be used more effectively, improving overall resource utilization.

[0070] This embodiment provides a resource scheduling method. When the editor is not started, a callback mechanism and a scheduled task are set. Through the callback mechanism, it is ensured that when the editor is recycled, its attached debugging container will also be automatically deleted and stopped, thereby avoiding the debugging container from still occupying valuable computing resources when the editor is not in use; triggering the debugging container to start the code debugging process through a pre-established communication tunnel, so that the debugging container is occupied only when code debugging is needed to avoid unnecessary computing power waste; determining the required initial computing power configuration according to the machine learning modeling code entering the code debugging process, based on the initial computing power configuration, applying for the computing power required for the corresponding debugging container to run, and realizing the reasonable allocation of computing power resources; in the code debugging process, monitoring the operation of the debugging container through a scheduled task, timely discovering idle computing power, once it is found that the debugging container is idle for more than the set threshold, deleting the debugging container and releasing the computing power resources occupied by it. In summary, this application reclaims the computing power of the debugging container attached to the editor in an inactive state by setting a callback mechanism, starts the debugging container on demand and allocates reasonable computing power resources, and timely detects and recycles idle computing power, avoids long-term idling and waste of resources, and effectively improves resource utilization and turnover rate.

[0071] In a feasible implementation, in the code debugging process, the step S40 of detecting and recovering idle computing power based on the scheduled task for the computing power required for running the debugging container may include steps S41 to S42:

[0072] Step S41, in the code debugging process, performing idle computing power detection on the computing power required for running the debugging container through the scheduled task, and calculating the idle time of the debugging container;

[0073] Create a scheduled task in the editor, which is executed at a predetermined time interval (such as every minute, every hour, etc.). The scheduled task monitors the code debugging process in the debugging container associated with the editor through API calls and log analysis.

[0074] The idle time of the debugging container refers to the time from when the debugging process stops running (or reaches a certain idle state) to when it is restarted next time or a new task is executed. The scheduled task checks the state of the debugging container every time it is executed, and calculates and records the idle time.

[0075] Step S42: If the idle time of the debugging container exceeds a preset threshold, the debugging container is automatically deleted and stopped, and the idle computing power required for running the debugging container is recovered.

[0076] The machine learning modeling system sets a preset idle time threshold, which is usually determined based on factors such as resource utilization and cost-effectiveness. If the scheduled task finds that the idle time of a debugging container exceeds the preset threshold, the machine learning modeling system will automatically delete and stop the debugging container.

[0077] In this embodiment, by creating a scheduled task to monitor the code debugging process of the debugging container, the machine learning modeling system can understand the resource usage in real time. This dynamic monitoring helps the system to promptly discover and process idle resources, thereby optimizing resource allocation and improving overall resource utilization.

[0078] Based on the first embodiment of the present application, a second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the first embodiment can be referred to the above introduction, and will not be described in detail later.

[0079] In this embodiment, when the startup status of the editor is not started, a callback mechanism and a scheduled task are set in the editor, and the debugging container corresponding to the editor is automatically deleted and stopped by the callback mechanism to recover computing power before step S10, step A01 is also included:

[0080] Step A01, separating the running computing power of the editor and the debugging container.

[0081] Since the editor is mainly used to write and modify code, and the debugging container is used to run and debug code. In the traditional development method, the editor and the debugging environment are often closely integrated, which may cause the editor and the debugging environment to compete for the same computing resources, resulting in performance degradation. Therefore, in the resource scheduling method provided in this application, the editor and the debugging container are separated in computing power.

[0082] To run the editor and debugging containers for computing power analysis, you first need to select container cloud technologies that can provide a lightweight, portable, and isolated operating environment, such as Docker, Kubernetes, etc.

[0083] Furthermore, according to the requirements of the editor, an IDE host container is selected to run the editor, appropriate computing power (such as CPU, memory) and services (such as network connection, file system access, etc.) are allocated to the container, and the dependencies required for the normal operation of the editor are installed in the container.

[0084] Furthermore, according to the requirements of the debugging environment, higher computing power (such as more CPU cores, larger memory) and necessary services (such as debugging tools, database connections, etc.) are allocated to the debugging container, and necessary dependencies are installed in the debugging container.

[0085] Exemplarily, a code transfer mechanism can be configured in the editor and the debugging container to transfer the code in the editor to the debugging container. Alternatively, a version control system (such as Git) can be used to track and manage code changes and synchronize code between the editor container and the debugging container.

[0086] In this embodiment, by placing the editor and debugging operations in different containers, different computing power and services are assigned to different containers, and the computing power for debugging the code is also provided by the container, the separation of the running computing power of the editor and the debugging container is achieved, making it possible to start containers that occupy more computing power resources on demand, providing a basis for subsequent computing power recovery.

[0087] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the first embodiment and / or the second embodiment can be referred to the above introduction, and will not be repeated in the following.

[0088] On this basis, please refer to Figure 2 After step S10, the resource scheduling method further includes steps S01 to S02:

[0089] Step S01, starting an editor provided with a callback mechanism and a scheduled task according to a pre-acquired editor start instruction;

[0090] The machine learning modeling system starts the editor that has been set up with a callback mechanism and a scheduled task by obtaining the editor startup instruction issued by the user. In other words, the editor in the startup state is set up with a callback mechanism and a scheduled task.

[0091] Step S02, capturing interactive signals through an editor that is set with a callback mechanism and a scheduled task in a startup state to determine the machine learning modeling code.

[0092] Once the editor is up and running, it begins capturing interactive signals from user input, such as keyboard input, mouse clicks, file open requests, etc. The machine learning modeling system receives the interactive signals through its user interface (UI) or application program interface (API) and determines the machine learning modeling code that the user is editing or running based on the type and content of the signal.

[0093] Exemplarily, this application uses Jupyter Kernel to execute user-submitted code and process input, output, and control messages during code execution. As one of the core components of Jupyter Notebook, Jupyter Kernel allows users to write code in the Notebook front end and execute it in the background through Kernel to view the execution results of the code in real time. In the machine learning modeling system, Kernel is associated with the debugging container to provide users with an interactive code running environment.

[0094] In this embodiment, an editor configured with a callback mechanism and a scheduled task is started through an editor startup instruction, and the machine learning modeling code that the user is editing or running is determined by capturing interactive signals, providing a basis for subsequently triggering the debugging container to start the code debugging process and the initial computing power configuration.

[0095] Based on the above embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0096] In this embodiment, before step S20 of triggering the debugging container to start the code debugging process through the pre-established communication tunnel, steps S1 to S3 are also included:

[0097] Step S1, generating an encrypted public key according to the port forwarding protocol of the editor, and writing the encrypted public key into a public configuration directory;

[0098] The editor communicates with the debug container through the SSH port forwarding protocol, which defines how data is transmitted between the editor and the debug container. To ensure the security of communication, the editor uses the RSA encryption algorithm to generate a pair of public and private keys. The public key is used to encrypt data sent to the debug container to ensure that the data is not stolen or tampered with during transmission.

[0099] The generated encrypted public key is written to a common configuration directory. This common configuration directory is accessible to all related components (such as editors, debug containers, etc.). The location of the common configuration directory is located in the editor and debug container shared files.

[0100] Step S2, reading the encryption public key from the public configuration directory to configure the encryption server of the debugging container;

[0101] The debug container reads the encryption public key generated by the editor from the public configuration directory when it starts, and uses the read public key to configure its encryption server, which means that the debug container is now able to use the same encryption algorithm to decrypt the encrypted data received from the editor.

[0102] Step S3, writing the debugging container IP into the public configuration directory according to the debugging container encryption server, and establishing a communication tunnel between the editor and the debugging container.

[0103] Once the encryption server configuration of the debug container is complete, the machine learning modeling system writes the debug container IP address to the public configuration directory, where the debug container IP address is the target address used by the editor to establish the SSH communication tunnel. The editor reads the debug container IP address from the public configuration directory and uses the previously configured port forwarding protocol and encryption public key to establish a communication tunnel with the debug container.

[0104] It should be noted that the communication tunnel can be direct (such as a TCP connection) or indirect (such as through an intermediate proxy or load balancer). The establishment of the tunnel ensures that data can be transmitted securely and reliably between the editor and the debugging container.

[0105] In this embodiment, the editor (IDE host container) uses the SSH port forwarding protocol to ensure secure and efficient communication with the debugging container. This protocol allows data to be transmitted between different containers as if it were flowing within the same machine, reducing network latency and improving data transmission security. It provides a basis for the subsequent code debugging process of the debugging container to be triggered through the communication tunnel.

[0106] Based on the above embodiments of the present application, a fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0107] In this embodiment, the step S20 of triggering the debugging container to start the code debugging process through the pre-established communication tunnel may include steps S21-S22:

[0108] Step S21, obtaining a code debugging instruction issued by a user through the communication tunnel;

[0109] Once the communication tunnel is established, the editor can send commands or signals to the debugging container through the tunnel to trigger the start of the code debugging process. These commands or signals include debugging configuration information (such as breakpoints, monitoring variables, etc.), code files to be debugged, etc.

[0110] The user performs debugging operations in the editor, such as setting breakpoints, starting debugging, single-step execution, etc. These operations generate corresponding code debugging instructions. The editor sends these debugging instructions to the debugging container through the previously established communication tunnel. The tunnel ensures that the instructions will not be stolen, tampered with, or lost during transmission. The debugging container receives the debugging instructions from the editor at the other end of the communication tunnel. These instructions are parsed and prepared for subsequent debugging processes.

[0111] Exemplarily, after receiving a command or signal from the editor, the debugging container starts executing the code debugging process. This may involve steps such as loading code, setting breakpoints, executing code, and catching exceptions. Information during the debugging process (such as variable values, execution status, etc.) is returned to the editor in real time through the communication tunnel so that the user can view and interact in the editor.

[0112] Step S22: triggering the debugging container to enter the code debugging process according to the code debugging instruction.

[0113] After receiving the instructions, the debugging server inside the debugging container first parses these instructions. The parsing process includes identifying the instruction type (such as starting debugging, setting breakpoints, etc.) and extracting the parameters carried in the instruction (such as breakpoint location, code file to be debugged, etc.). According to the parsing results, the code debugging process is entered.

[0114] In this embodiment, the debugging container is triggered to enter the code debugging process through a pre-established communication tunnel, so that the debugging container is started only when needed, avoiding idling and waste of resources and improving the utilization rate of computing resources.

[0115] Based on the above embodiments of the present application, a sixth embodiment of the present application is proposed. In the sixth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0116] In this embodiment, the step S30 of applying for the computing power required for the debugging container operation based on the pre-acquired computing power configuration instruction may include steps S31 to S33:

[0117] Step S31, obtaining a computing power configuration instruction and monitoring whether the user inputs a computing power adjustment instruction;

[0118] The machine learning modeling system obtains an initial computing power configuration instruction from the user input. This instruction usually contains the basic computing power requirements required for debugging container operation, such as the number of CPU cores and memory size. While obtaining the computing power configuration instruction, the machine learning modeling system starts a monitoring mechanism for continuously monitoring whether the user has entered the computing power adjustment instruction. This monitoring mechanism can be implemented in many ways, such as monitoring a specific user input interface, checking changes in user configuration files, etc.

[0119] It should be noted that the computing power adjustment instruction is input by the user according to actual needs or performance optimization requirements, and is used to dynamically adjust the computing power configuration of the debugging container.

[0120] Step S32: if a computing power adjustment instruction input by the user is detected, applying for the computing power required for the debugging container to run according to the computing power adjustment instruction;

[0121] When the machine learning modeling system detects the computing power adjustment instructions input by the user, it will adjust the computing power configuration of the debugging container according to the computing power adjustment instructions. Specifically, the machine learning modeling system will parse the parameters in the computing power adjustment instructions, such as the new number of CPU cores, memory size, etc., and apply for the corresponding computing power resources from the underlying resource management system (such as Kubernetes, Docker Swarm, etc.) based on these parameters.

[0122] During the application process, the machine learning modeling system may perform a series of checks and verifications to ensure that the computing resources applied for are legal and available. Once the application is successful, the machine learning modeling system will update the computing configuration of the debugging container and notify the user or related development environment.

[0123] Step S33: If the computing power adjustment instruction input by the user is not detected, the computing power required for the debugging container operation is applied for according to the computing power configuration instruction.

[0124] If the machine learning modeling system does not detect the computing power adjustment instructions input by the user within a certain period of time, it will apply for the computing power resources of the debugging container based on the computing power configuration instructions initially obtained, mainly by parsing the parameters in the computing power configuration instructions and applying for the corresponding computing power resources from the underlying resource management system. For example, the machine learning modeling system can also automatically trigger the computing power adjustment process periodically or according to specific conditions (such as performance bottlenecks, excessive resource utilization, etc.) to optimize the performance of the debugging container.

[0125] In this embodiment, by monitoring the computing power adjustment instructions input by the user and applying for corresponding computing power resources according to the computing power configuration instructions, it can flexibly adapt to different development needs and performance optimization requirements, improve the utilization rate of computing power resources, reduce development costs, and improve the overall performance of the development environment.

[0126] The following combination Figure 3 , Figure 4 The computing resources of the scheduling editor and debugging container in the resource scheduling method of this application are explained in detail. Figure 3 This is a logical diagram of computing resource scheduling for starting the editor. Figure 4 A logical diagram of computing resource scheduling for starting the debugging container.

[0127] like Figure 3 As shown in the figure, the IDE host container is selected as the editor. When the editor is not started, a preStop callback function is set. The purpose of the function is to automatically delete the debug container associated with it when the IDE host container is stopped. This ensures the effective recycling of resources.

[0128] Furthermore, a scheduled task is created in the IDE host container. This scheduled task will periodically monitor the code running process on the debug container and calculate and record the idle time of the debug container.

[0129] Furthermore, in order to establish a communication tunnel between the editor and the debugging container, the IDE host container generates an SSH public key and saves the public key in the public configuration directory. The generation and storage of the SSH public key is to ensure the subsequent secure communication with the debugging container or other services.

[0130] like Figure 4 As shown, if the debug container has not been started, the startup operation is performed. Specifically, the startup operation includes creating a container instance from the image, allocating necessary resources (such as CPU, memory, etc.), setting up the network, etc. In the process of starting the debug container, the SSH public key generated by the editor is read from the public configuration directory, the debug container SSH service port is configured so that the editor can SSH to the debug container without a password, the debug container IP address is written to the public configuration directory, the debug container idle time is reset, and the SSH communication tunnel between the editor and the debug container is created accordingly.

[0131] Furthermore, a debugging Kernel session process is started inside the debugging container to perform code debugging. During the code debugging process, the computing power configuration of the debugging container is modified in combination with the computing power configuration modification instruction input by the user to improve the utilization rate of computing power resources.

[0132] This application also provides a resource scheduling device, please refer to Figure 5 , the resource scheduling device comprises:

[0133] The computing power recovery module 10, when the startup status of the editor is not started, sets a callback mechanism and a scheduled task in the editor, and automatically deletes and stops the debugging container corresponding to the editor through the callback mechanism to recover computing power;

[0134] The communication module 20 is used to trigger the debugging container to start the code debugging process through a pre-established communication tunnel;

[0135] A computing power configuration module 30, used to apply for the computing power required for the debugging container operation based on the computing power configuration instructions obtained in advance;

[0136] The idle computing power recovery module 40 is used to detect and recover the idle computing power required for running the debugging container based on the scheduled task during the code debugging process.

[0137] The resource scheduling device provided by the present application adopts the resource scheduling method in the above embodiment to solve the technical problem of resource scheduling. Compared with the prior art, the beneficial effects of the resource scheduling device provided by the present application are the same as the beneficial effects of the resource scheduling method provided by the above embodiment, and other technical features in the resource scheduling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0138] The present application provides a resource scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the resource scheduling method in the above-mentioned embodiment 1.

[0139] Reference below Figure 6 , which shows a schematic diagram of the structure of a resource scheduling device suitable for implementing the embodiment of the present application. The resource scheduling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The resource scheduling device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0140] like Figure 6As shown, the resource scheduling device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the resource scheduling device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the resource scheduling device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a resource scheduling device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0141] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0142] The resource scheduling device provided by the present application adopts the resource scheduling method in the above embodiment to solve the technical problem of resource scheduling. Compared with the prior art, the beneficial effects of the resource scheduling device provided by the present application are the same as the beneficial effects of the resource scheduling method provided by the above embodiment, and other technical features in the resource scheduling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0144] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0145] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the resource scheduling method in the above-mentioned embodiment.

[0146] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0147] The computer-readable storage medium may be included in the resource scheduling device; or may exist independently without being assembled into the resource scheduling device.

[0148] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the resource scheduling device, the resource scheduling device: when the startup status of the editor is not started, sets a callback mechanism and a scheduled task in the editor, and automatically deletes and stops the debugging container corresponding to the editor through the callback mechanism to recycle computing power; triggers the debugging container to start the code debugging process through a pre-established communication tunnel; applies for the computing power required for the debugging container to run based on the pre-acquired computing power configuration instructions; and in the code debugging process, performs idle computing power detection and recovery on the computing power required for the debugging container to run based on the scheduled task.

[0149] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0152] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned resource scheduling method, and can solve the technical problem of resource scheduling. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the resource scheduling method provided in the above-mentioned embodiment, and will not be repeated here.

[0153] The present application also provides a computer program product, including a computer program, which implements the steps of the resource scheduling method as described above when executed by a processor.

[0154] The computer program product provided by this application can solve the technical problem of resource scheduling. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the resource scheduling method provided by the above embodiment, which will not be repeated here.

[0155] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A resource scheduling method, characterized in that: Applied to a machine learning modeling system, the machine learning modeling system includes an editor and a debugging container, the method includes: When the startup status of the editor is not started, a callback mechanism and a scheduled task are set in the editor, and the debugging container corresponding to the editor is automatically deleted and stopped through the callback mechanism to recycle computing power; Triggering the debugging container to start a code debugging process through a pre-established communication tunnel; Apply for the computing power required for debugging container operation based on the computing power configuration instructions obtained in advance; In the code debugging process, idle computing power detection and recovery are performed on the computing power required for running the debugging container based on the scheduled task.

2. The method according to claim 1, characterized in that Before the step of setting a callback mechanism and a scheduled task in the editor when the startup status of the editor is not started, and automatically deleting and stopping the debugging container corresponding to the editor through the callback mechanism to recover computing power, the step also includes: The editor and the debugging container are operated with separate computing power.

3. The method according to claim 1, characterized in that After the step of setting a callback mechanism and a scheduled task in the editor when the startup status of the editor is not started, and automatically deleting and stopping the debugging container corresponding to the editor through the callback mechanism to recover computing power, the step further includes: Starting an editor with a callback mechanism and a scheduled task according to a pre-acquired editor startup instruction; Capture interactive signals to identify machine learning modeling code through an editor with callback mechanisms and scheduled tasks set up in the startup state.

4. The method according to claim 1, characterized in that Before the step of triggering the debugging container to start the code debugging process through the pre-established communication tunnel, the step further includes: Generate an encrypted public key according to the port forwarding protocol of the editor, and write the encrypted public key into a public configuration directory; Read the encryption public key from the public configuration directory to configure the encryption server of the debugging container; The debugging container encryption server writes the debugging container IP into a public configuration directory, and establishes a communication tunnel between the editor and the debugging container.

5. The method according to claim 3, characterized in that The step of triggering the debugging container to start the code debugging process through the pre-established communication tunnel includes: Acquire a code debugging instruction issued by a user through the communication tunnel; The debugging container is triggered to enter a code debugging process according to the code debugging instruction.

6. The method according to claim 1, characterized in that The step of applying for the computing power required for running the debugging container based on the pre-acquired computing power configuration instruction includes: Obtain computing power configuration instructions and monitor whether the user enters computing power adjustment instructions; If a computing power adjustment instruction input by the user is detected, the computing power required for the debugging container operation is applied according to the computing power adjustment instruction; If the computing power adjustment instruction input by the user is not detected, the computing power required for the debugging container operation is applied for according to the computing power configuration instruction.

7. The method according to claim 1, characterized in that In the code debugging process, the steps of detecting idle computing power and recovering idle computing power based on the scheduled task for the computing power required for running the debugging container include: In the code debugging process, the computing power required for running the debugging container is detected to be idle through the scheduled task, and the idle time of the debugging container is calculated; If the debugging container is idle for a period of time exceeding a preset threshold, the debugging container is automatically deleted and stopped, and the idle computing power required for running the debugging container is recovered.

8. A resource scheduling device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the resource scheduling method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the resource scheduling method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the resource scheduling method according to any one of claims 1 to 7 are implemented.

Images