Online algorithm development methods, systems, and equipment for dynamic management of runtime environment

Abstract

This invention discloses an online algorithm development method, system, and device for dynamic management of the runtime environment. Addressing the problem that current algorithm development platforms often have pre-installed algorithm components, making it difficult to expand and flexibly support rapidly changing business scenarios, this invention establishes a basic algorithm development environment, constructs algorithm components, uploads the required dependency files for the algorithm components, and obtains an environment library dependency list. It then creates an algorithm runtime environment container, selects programming language versions and corresponding dependency files from the environment library dependency list, and establishes an extended algorithm runtime environment. The extended algorithm is coded, debugged, and deployed online using an algorithm editor. During task modeling, multiple algorithm components are selected, and the training and runtime environments of the algorithm components are dynamically managed. This provides an environment expansion scheme, generating extended algorithm environments based on selected dependency packages to achieve algorithm environment expansion.

Description

Technical Field

[0001] This invention belongs to the technical field of data mining, and in particular relates to an online algorithm development method, system and device for dynamic management of the operating environment. Background Technology

[0002] Currently, algorithm development platforms on the market are all developing towards lower barriers to entry and higher reusability. Improving the efficiency of algorithm development and application has become paramount and is also the main research direction for various platforms. For example, Chinese patent CN108279890B, entitled "Component Deployment Method, Component Construction Method, and Graphical Machine Learning Algorithm Platform," provides a component deployment method, a component construction method, and a graphical machine learning algorithm platform. Graphical machine learning algorithm platforms can deploy functional models or build them into new components. Therefore, when users need to reuse a certain function, they can directly use the new component without having to repeatedly build the functional model, thus facilitating user use. However, these components are pre-installed and difficult to expand and flexibly support rapidly changing business scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide an online algorithm development method, system, and device for dynamic management of the operating environment, which greatly improves the development and operation efficiency of platform-based algorithm expansion by lowering the threshold of algorithm development through platformization.

[0004] To solve the above problems, the technical solution of the present invention is as follows:

[0005] An online algorithm development method with dynamic runtime environment management, comprising:

[0006] Set up the basic environment for algorithm development. The basic environment includes programming language version files and basic dependency files, and isolate the files in the basic environment.

[0007] Build the algorithm component and upload the dependency files required by the algorithm component; during the upload of dependency files, establish a virtual environment according to the selected programming language version and verify the installation of the uploaded dependency files; after successful installation, summarize the dependency files and establish a dependency relationship graph to obtain the environment library dependency list.

[0008] Create an algorithm runtime environment container, select the programming language version and corresponding dependency files from the environment library dependency list, and establish an extended algorithm runtime environment;

[0009] The extended algorithm can be coded and debugged online using the algorithm editor;

[0010] Integrate the debugged extended algorithm code into the extended algorithm runtime environment and release it for operation;

[0011] When modeling a task, multiple algorithm components are selected, and the algorithm environment in which the algorithm components reside is dynamically managed, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling.

[0012] According to one embodiment of the present invention, after successful installation, the virtual environment is destroyed after the dependency files are summarized and a dependency graph is established.

[0013] According to one embodiment of the present invention, when uninstalling dependent files, the dependency relationship graph is used for verification. If the dependent file to be uninstalled has a dependency relationship with other dependent files or is associated with other algorithms, it cannot be uninstalled.

[0014] If there are multiple alternative dependency files for the dependency file to be uninstalled, uninstallation is allowed until at least one alternative dependency file exists.

[0015] According to an embodiment of the present invention, the online coding and debugging of the extended algorithm through an algorithm editor further includes:

[0016] Predefine built-in variables during the algorithm development process for data interaction with upstream and downstream algorithm businesses.

[0017] According to an embodiment of the present invention, when performing real-time debugging during online encoding, it is necessary to start a first process and a second process, wherein the first process is used for the current algorithm environment and the second process is used to connect to the extended algorithm environment; and a bidirectional communication is established between the first process and the second process, including read and write dual channels;

[0018] The algorithm execution code and the built-in variables are transferred from the first process to the second process for execution via a write channel; during execution, execution process information and exception error information are read from the second process to the first process via a read channel and persisted to a local file for storage.

[0019] Meanwhile, the execution process information and error messages are printed to the console via WebSocket for front-end display, which is used for debugging and troubleshooting.

[0020] According to an embodiment of the present invention, the dynamic management of the training environment for task modeling further includes:

[0021] The algorithm environment is launched by integrating multiple algorithm components required by the task model into one environment container, and the multiple algorithm environments are used in a serial order;

[0022] Once the task model has finished training, destroy the environment container and release resources.

[0023] According to an embodiment of the present invention, the runtime environment dynamic management for task modeling further includes:

[0024] Each algorithm component required by the task model starts its algorithm environment in a separate environment container. Each environment container has n processes running as needed to receive new algorithm tasks. Each process preloads the algorithm dependency files, executes them immediately after receiving upstream algorithm business data, and returns the results.

[0025] An online algorithm development system with dynamic runtime environment management includes:

[0026] A basic environment platform is used to build the basic environment for algorithm development. The basic environment includes programming language version files and basic dependency files, and the files in the basic environment are isolated.

[0027] The algorithm component module is used to build algorithm components and upload the dependency files required by the algorithm components. During the upload of dependency files, a virtual environment is established according to the selected programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency relationship graph is established to obtain the environment library dependency list.

[0028] An environment container is used to create an algorithm runtime environment container. It selects the programming language version and corresponding dependency files from the environment library dependency list to establish an extended algorithm runtime environment.

[0029] An algorithm editor for online coding and debugging of extended algorithms;

[0030] The algorithm execution module is used to integrate the debugged extended algorithm code into the extended algorithm runtime environment for deployment and execution.

[0031] The dynamic management module is used to select multiple algorithm components during task modeling and to dynamically manage the algorithm environment in which the algorithm components reside, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling.

[0032] An online algorithm development device for dynamic management of runtime environment includes: a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the online algorithm development method for dynamic management of runtime environment according to an embodiment of the present invention.

[0033] A storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform steps in an online algorithm development method for dynamic management of the runtime environment according to an embodiment of the present invention.

[0034] Because of the above technical solutions, this invention has the following advantages and positive effects compared with the prior art:

[0035] 1) An online algorithm development method for dynamic management of the runtime environment according to one embodiment of the present invention addresses the problem that current algorithm development platforms often have pre-installed algorithm components, making it difficult to expand and flexibly support rapidly changing business scenarios. This method establishes a basic environment for algorithm development, constructs algorithm components, and uploads the dependency files required by the algorithm components. During the dependency file upload process, a virtual environment is established based on the selected programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency graph is established to obtain an environment library dependency list. An algorithm runtime environment container is created, and programming language versions and corresponding dependency files are selected from the environment library dependency list to establish an extended algorithm runtime environment. The extended algorithm is coded and debugged online using an algorithm editor. The debugged extended algorithm code is integrated into the extended algorithm runtime environment and released for execution. During task modeling, multiple algorithm components are selected, and the algorithm environment in which the algorithm components reside is dynamically managed, including dynamic management of the training environment and the runtime environment for task modeling. The above provides an environment expansion scheme, generating extended algorithm environments based on selected dependency packages to achieve algorithm environment expansion.

[0036] 2) In one embodiment of the present invention, the online algorithm development method for dynamic management of the operating environment achieves effective integration with upstream and downstream business data during the online coding process by pre-setting built-in variables.

[0037] 3) The online algorithm development method for dynamic management of the operating environment in one embodiment of the present invention enables rapid location of coding anomalies during the algorithm development process through online coding and debugging.

[0038] 4) The online algorithm development method for dynamic management of the runtime environment in one embodiment of the present invention achieves effective control of algorithm runtime resources and reduces system risks through virtualization deployment and dynamic startup and destruction of the environment. Attached Figure Description

[0039] Figure 1 This is a flowchart of an online algorithm development method for dynamic management of the runtime environment in one embodiment of the present invention;

[0040] Figure 2 This is a functional architecture diagram of an online algorithm development system for dynamic management of the runtime environment according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of an online algorithm editor in one embodiment of the present invention;

[0042] Figure 4 This is a configuration interface diagram of the task model algorithm in one embodiment of the present invention;

[0043] Figure 5This is a block diagram of an online algorithm development system for dynamic management of the runtime environment according to an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of an online algorithm development device for dynamic management of the operating environment in one embodiment of the present invention. Detailed Implementation

[0045] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more detailed account of the online algorithm development method, system, and device for dynamic management of the operating environment proposed in this invention. The advantages and features of this invention will become clearer from the following description and claims.

[0046] Example 1

[0047] Please refer to Figure 1 This embodiment provides an online algorithm development method with dynamic management of the runtime environment, including the following steps:

[0048] S1: Set up the basic environment for algorithm development. This basic environment includes programming language version files and basic dependency files, and isolate the files in the basic environment.

[0049] S2: Build algorithm components and upload the dependency files required by the algorithm components; during the process of uploading dependency files, establish a virtual environment according to the programming language version and verify the installation of the uploaded dependency files; after successful installation, summarize the dependency files and establish a dependency graph to obtain the environment library dependency list.

[0050] S3: Create an algorithm runtime environment container, select the programming language version and corresponding dependency files from the environment library dependency list, and establish an extended algorithm runtime environment;

[0051] S4: Online coding and debugging of extended algorithms via the algorithm editor;

[0052] S5: Integrate the debugged extended algorithm code into the extended algorithm runtime environment and release it for operation;

[0053] S6: When modeling a task, select multiple algorithm components and dynamically manage the algorithm environment in which the algorithm components are located, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling.

[0054] In step S1, the basic environment for algorithm development is set up, which includes programming language version files and basic dependency files, and the files in the basic environment are isolated.

[0055] The foundational environment for algorithm development includes both hardware and software environments. Typically, the hardware environment, such as a server, includes basic computing components like CPU, GPU, memory, operating system, network configuration, and disk. The software environment, depending on the domain requirements, includes pre-configured software components that refine application tasks and are implemented through granular computation, allowing users to easily build their own applications. These software components, such as a Python system (e.g., Python 2.7.x or Python 3.x), are installed and upgraded using terminal commands like apt, pip, and git, which are used to install or upgrade related software and dependencies.

[0056] In step S2, the algorithm component is constructed, and the dependency files required by the algorithm component are uploaded. During the upload of dependency files, a virtual environment is established according to the programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency relationship graph is established to obtain the environment library dependency list.

[0057] After successful installation, the system compiles the dependency files and establishes a dependency graph, then destroys the virtual environment and releases resources.

[0058] When uninstalling dependent files, the dependency graph is used for verification. If the dependent file to be uninstalled has a dependency relationship with other dependent files or is associated with other algorithms, it cannot be uninstalled.

[0059] If there are multiple alternative dependency files for the dependency file to be uninstalled, uninstallation is allowed until at least one alternative dependency file exists.

[0060] In step S3, an algorithm runtime environment container is created by selecting the programming language version and corresponding dependency files from the environment library dependency list to establish an extended algorithm runtime environment.

[0061] Containers serve as an alternative to virtual machines, helping developers build, migrate, deploy, and instantiate applications. A container is a collection of processes that share an operating system instance but operate independently of other processes on the server system. Containers do not require a full operating system, making them more lightweight than virtual machines. Because containers can start in seconds, they can scale to meet application needs with minimal resource allocation.

[0062] Existing container solutions primarily utilize Docker, a virtualization technology built upon the lightweight virtualization mechanism LXC (Linux Containers). It's a lightweight virtualization solution based on process containers. Docker uses Docker containers as the basic unit for resource partitioning and scheduling, encapsulating the entire software runtime environment. Designed for developers and system administrators, it provides a platform for building, deploying, and running distributed applications. Docker's most significant advantage over other virtualization technologies is its fast startup and low resource consumption. Therefore, for building isolated, standardized runtime environments, lightweight PaaS, automated testing and continuous integration environments, and all horizontally scalable applications (especially web applications requiring rapid startup and shutdown to handle peak and off-peak conditions), Docker is a cross-platform, portable, and easy-to-use container solution.

[0063] When dealing with graphical user interface applications using the Windows operating system, application containers based on the Windows operating system can be used. These containers can be implemented using application layer compatibility engines, memory isolation engines, file isolation engines, registry isolation engines, and network isolation engines, making them compatible with multiple Windows operating systems.

[0064] In practical applications, Docker container technology or application containers that are compatible with multiple Windows operating systems can be used as needed to create an extended algorithm runtime environment.

[0065] In step S4, the extended algorithm is coded and debugged online using an algorithm editor.

[0066] When coding and debugging extended algorithms online through the algorithm editor, predefined built-in variables are used for data linkage with upstream and downstream algorithm business processes.

[0067] When performing real-time debugging during online coding, it is necessary to start a first process and a second process. The first process is used for the current algorithm environment, and the second process is used to connect to the extended algorithm environment. A two-way communication is established between the first process and the second process, including read and write dual channels.

[0068] The algorithm execution code and built-in variables are transferred from the first process to the second process for execution via a write channel. During execution, execution process information and exception error information are read from the second process to the first process via a read channel and persisted to a local file for storage.

[0069] Meanwhile, the execution process information and error messages are printed to the console via a websocket tool for front-end display, which is used for debugging and troubleshooting.

[0070] In step S5, the debugged extended algorithm code is integrated into the extended algorithm runtime environment and released for execution.

[0071] Newly developed extension algorithms need to be debugged, which is called algorithm training. After the algorithm is successfully debugged, the algorithm code and its runtime environment are packaged into an image using container technology and released as a production runtime environment container.

[0072] In step S6, during task modeling, multiple algorithm components are selected, and the algorithm environment in which the algorithm components reside is dynamically managed, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling.

[0073] In the dynamic management of the training environment for task modeling, multiple algorithm components required by the task model are concentrated in one environment container to start the algorithm environment, and multiple algorithm environments are used in a serial order; when the training of the task model is completed, the environment container is destroyed and resources are released.

[0074] When dynamically managing the runtime environment for task modeling, each algorithm component required by the task model is started in a separate environment container. Each environment container has n resident processes as needed to receive new algorithm tasks. Each process preloads algorithm dependency files, executes them immediately after receiving upstream algorithm business data, and returns the results.

[0075] The online algorithm development method with dynamic runtime environment management in this embodiment addresses several issues. First, in terms of environment expansion, it flattens all environment dependencies, establishing a unified dependency management platform. Different algorithms can establish corresponding runtime environments as needed, while also providing corresponding environment and dependency management functions. Second, regarding business integration, it efficiently combines data from the business system with the extended algorithm by pre-setting intermediate variables in the algorithm. Third, in terms of algorithm development efficiency, it provides an online development editor, real-time console debugging, log output, and rapid error location. Fourth, in terms of system stability, it effectively controls resource usage through virtualization deployment and dynamic environment startup and destruction. Overall, this invention significantly improves the development and operational efficiency of platform-based algorithm extensions by lowering the barrier to algorithm development through platformization.

[0076] Application examples

[0077] Please refer to Figure 2 and 3 The following example uses Python as the programming language for algorithm development to demonstrate an online algorithm development method with dynamic runtime environment management:

[0078] 1. Upload dependency files to centrally manage the environment and dependency files.

[0079] (1) First, the system has multiple mainstream versions of the Python programming language (such as 2.7, 3.5, 3.7, etc.). All environments are official environments, with only basic dependencies built in. The system also isolates the basic environment files through file isolation.

[0080] (2) Then, using the Python package management tool (pip), find, download, and prepare the required Python dependency packages and the corresponding requirements.txt file (used to record the dependency names and version information to be installed). The main information of requirements.txt is shown in the table below.

[0081] name Version matplotlib 3.2.2 numpy 1.19.5 pandas 1.0.5 scipy 1.5.1 ... ...

[0082] (3) During the upload process, a virtual environment is created based on the selected programming language version. The uploaded dependency packages are verified for installation. Then, the installation information is displayed in a rolling manner, and finally, the installation is indicated as successful or unsuccessful. After successful installation, the names, version information, and dependency relationships of each dependency package are summarized, and a dependency relationship graph is created. The virtual runtime environment is then destroyed after the process is complete.

[0083] (4) When uninstalling dependencies, the dependency graph established in the previous step is used for verification. If other dependency packages or associated algorithms have dependencies, the current dependency package cannot be deleted. If the current dependency package has multiple alternative dependency packages, deletion is allowed until at least one alternative dependency package exists.

[0084]

[0085] 2. Create an algorithm runtime environment container, install the selected dependency files, and bind the environment to the integrated development environment after successful installation.

[0086] (1) For the built-in algorithms, since the built-in algorithms are all pre-developed, the corresponding built-in environment will be pre-integrated separately in the system (different from the subsequent extended algorithm environment). All built-in algorithms run in a unified built-in environment, which makes them more stable and controllable.

[0087] (2) For extended algorithms, each extended algorithm has its own separate running environment. To create it, you need to select the built-in Python version, select the required dependency files from the preset environment library dependency list, and create the corresponding extended algorithm running environment for use as the subsequent extended algorithm integration development environment.

[0088] 3. Online algorithm development is divided into training and runtime phases, and intermediate variables (built-in variables) are pre-set.

[0089] (1) For algorithm development, an online algorithm development editor is provided. Here, we define the complete development process as a training period and a running period, which correspond to the two stages of training the model and using the model, respectively.

[0090] (2) For the training / runtime development process, predefine built-in variables for data linkage with upstream and downstream businesses on the platform. As shown in the table below, more fields can be added or removed as needed. Therefore, the above variables can be used directly in algorithm coding. The content of the 'model' field is used to store model information, so the specific usage depends on the corresponding algorithm developer.

[0091]

[0092] 4. Online coding and debugging of algorithm business code. Please see the editing page. Figure 4 .

[0093] (1) Real-time debugging is performed during the coding process. The specific implementation path is as follows: In addition to process A in the current business environment, another process B needs to be started to connect to the specified running environment created in step 2 of the extended algorithm, and to establish bidirectional communication between processes A and B, including read and write dual channels.

[0094] (2) The written algorithm script and the built-in variables defined above are passed from process A to process B for execution through the write channel; during the execution, the execution process information, exception error information, etc. are read from process B to process A through the read channel and persisted to a local file for storage, and a periodic clearing strategy is set.

[0095] (3) During the reading process, in addition to local storage, the information read is also printed in the IDE console via WebSocket technology, displaying the output information and error messages during the process for debugging and troubleshooting.

[0096] (4) During the debugging process, you can click the terminate button at any time to kill the B process immediately, thereby achieving the termination function. If you need to debug again, repeat step (2).

[0097] If incomplete environment dependency selection or incorrect version selection occurs during debugging, you can return to step 2 to specify the runtime environment for the newly created extended algorithm.

[0098] 5. After the algorithm is successfully debugged, the algorithm's business logic code is integrated into the runtime environment and released for operation.

[0099] (1) After the algorithm is debugged and passed, the algorithm code and Python runtime environment are packaged into an image using Docker virtualization and released as a production runtime environment container to achieve environment isolation for each extended algorithm. The debugging environment resources used in step 4 are then destroyed.

[0100] (2) Add the necessary business code to the formal running environment container of each algorithm to meet the corresponding business requirements, such as defining the call interface for algorithm execution.

[0101] 6. Dynamic management of the experimental task environment.

[0102] (1) Steps 1-5 above describe the complete creation process of a single algorithm. In fact, a single algorithm can be a built-in algorithm or an extended algorithm, and each has a training environment container and a runtime environment container, corresponding to the training period and the runtime period, respectively.

[0103] (2) However, in practice, most complete modeling tasks involve multiple algorithmic components. A common experimental procedure is: Start -> Data Preprocessing -> Feature Engineering -> Machine Learning Modeling -> Model Evaluation -> End.

[0104] (3) For dynamic management of the training environment, since training and modeling have high computational resource requirements, each time the algorithm components required for the experiment are summarized by the experimental task, and these algorithm environments are started in a container. That is, one experimental task corresponds to one environment container, and the environment container contains the algorithm environment of all components in this experiment. The algorithm environments are used in a serial order. After the experiment is completed, the environment container is destroyed directly to release resources.

[0105] For dynamic management of the runtime environment, the runtime has low requirements for computing resources. Each algorithm component runs in a separate container. To meet high concurrency requirements, each container can have n processes running as needed to receive new algorithm tasks. Each process preloads algorithm dependency packages, executes immediately upon receiving upstream business data, and returns results. Simply connect the runtime environments of each component according to the component order of the experimental task.

[0106] Example 2

[0107] This embodiment provides an online algorithm development system for dynamic runtime environment management, implementing the online algorithm development method for dynamic runtime environment management in Embodiment 1 above. Please refer to... Figure 5 The online algorithm development system includes:

[0108] Basic environment platform 1 is used to build the basic environment for algorithm development. This basic environment includes programming language version files and basic dependency files, and the files in the basic environment are isolated.

[0109] Algorithm component module 2 is used to build algorithm components and upload the dependency files required by the algorithm components. During the process of uploading dependency files, a virtual environment is established according to the selected programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency relationship graph is established to obtain the environment library dependency list.

[0110] Environment container 3 is used to create an algorithm runtime environment container. It selects the programming language version and corresponding dependency files from the environment library dependency list to establish an extended algorithm runtime environment.

[0111] Algorithm Editor 4 is used for online coding and debugging of extended algorithms;

[0112] Algorithm execution module 5 is used to integrate the debugged extended algorithm code into the extended algorithm execution environment for deployment and execution;

[0113] The dynamic management module 6 is used to select multiple algorithm components during task modeling and to dynamically manage the algorithm environment in which the algorithm components are located, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling.

[0114] The functions and implementation methods of the above-mentioned basic environment platform 1, algorithm component module 2, environment container 3, algorithm editor 4, algorithm running module 5 and dynamic management module 6 are as described in Embodiment 1 above, and will not be repeated here.

[0115] Example 3

[0116] This embodiment provides an online algorithm development device with dynamic management of the runtime environment. Please refer to... Figure 6 The online algorithm development device 500, which dynamically manages the runtime environment, can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 510 (e.g., x86, ARM architecture processors, or FPGAs) and memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module comprising a series of instruction operations within the dynamically managed online algorithm development device 500.

[0117] Furthermore, the processor 510 can be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the online algorithm development device 500 with dynamic management of the runtime environment.

[0118] The online algorithm development device 500 with dynamic management of the operating environment may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Vista, etc.

[0119] Those skilled in the art will understand that Figure 6 The structure of the online algorithm development device with dynamic management of the runtime environment shown does not constitute a limitation on the online algorithm development device with dynamic management of the runtime environment. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0120] Another embodiment of the present invention also provides a computer-readable storage medium.

[0121] The computer-readable storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the online algorithm development device method for dynamic management of the runtime environment in Embodiment 1.

[0122] If the online algorithm development method for dynamic management of the runtime environment is implemented in the form of program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in software form. This computer software is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0124] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. An online algorithm development method for dynamic management of a runtime environment, characterized by, include: Set up the basic environment for algorithm development. The basic environment includes programming language version files and basic dependency files, and isolate the files in the basic environment. Build the algorithm component and upload the dependency files required by the algorithm component; During the process of uploading dependency files, a virtual environment is created based on the selected programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency graph is created to obtain the environment library dependency list. Create an algorithm runtime environment container, select the programming language version and corresponding dependency files from the environment library dependency list, and establish an extended algorithm runtime environment; The extended algorithm can be coded and debugged online using the algorithm editor; Integrate the debugged extended algorithm code into the extended algorithm runtime environment and release it for operation; When modeling a task, multiple algorithm components are selected, and the algorithm environment in which the algorithm components reside is dynamically managed, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling. The step of online coding and debugging of the extended algorithm through an algorithm editor further includes: Predefine built-in variables during the algorithm development process for data interaction with upstream and downstream algorithm business processes; When performing real-time debugging during online coding, it is necessary to start a first process and a second process. The first process is used for the current algorithm environment, and the second process is used to connect to the extended algorithm environment. A two-way communication is established between the first process and the second process, including read and write dual channels. The algorithm execution code and the built-in variables are transferred from the first process to the second process for execution via the write channel. During execution, execution process information and exception error information are read from the second process to the first process via the read channel and persisted to a local file for storage.

2. The online algorithm development method for dynamic management of a runtime environment according to claim 1, wherein, After successful installation, the virtual environment is destroyed after the dependency files are compiled and a dependency graph is established.

3. The online algorithm development method for dynamic management of the runtime environment as described in claim 2, characterized in that, When uninstalling dependent files, the dependency graph is used for verification. If the dependent file to be uninstalled has a dependency relationship with other dependent files or is associated with other algorithms, it cannot be uninstalled. If there are multiple alternative dependency files for the dependency file to be uninstalled, uninstallation is allowed until at least one alternative dependency file exists.

4. The online algorithm development method for dynamic management of the runtime environment as described in claim 1, characterized in that, During real-time debugging in online coding, execution process information and error messages are printed to the console via WebSocket for front-end display, facilitating debugging and troubleshooting.

5. The online algorithm development method for dynamic management of the runtime environment as described in claim 1, characterized in that, Dynamic management of the training environment for task modeling further includes: The algorithm environment is launched by integrating multiple algorithm components required by the task model into one environment container, and the multiple algorithm environments are used in a serial order; Once the task model has finished training, destroy the environment container and release resources.

6. The online algorithm development method for dynamic management of the runtime environment as described in claim 1, characterized in that, Dynamic management of the runtime environment for task modeling further includes: Each algorithm component required by the task model starts its algorithm environment in a separate environment container. Each environment container has n processes running as needed to receive new algorithm tasks. Each process preloads the algorithm dependency files, executes them immediately after receiving upstream algorithm business data, and returns the results.

7. An online algorithm development system with dynamic management of the runtime environment, characterized in that, include: A basic environment platform is used to build the basic environment for algorithm development. The basic environment includes programming language version files and basic dependency files, and the files in the basic environment are isolated. The algorithm component module is used to build algorithm components and upload the dependency files required by the algorithm components; During the process of uploading dependency files, a virtual environment is created based on the selected programming language version, and the uploaded dependency files are verified for installation. After successful installation, the dependency files are summarized and a dependency graph is created to obtain the environment library dependency list. An environment container is used to create an algorithm runtime environment container. It selects the programming language version and corresponding dependency files from the environment library dependency list to establish an extended algorithm runtime environment. An algorithm editor for online coding and debugging of extended algorithms; The algorithm execution module is used to integrate the debugged extended algorithm code into the extended algorithm runtime environment for deployment and execution. The dynamic management module is used to select multiple algorithm components during task modeling and dynamically manage the algorithm environment in which the algorithm components are located, including dynamic management of the training environment and dynamic management of the runtime environment for task modeling. The algorithm editor is further configured as follows: Predefine built-in variables during the algorithm development process for data interaction with upstream and downstream algorithm business processes; When performing real-time debugging during online coding, it is necessary to start a first process and a second process. The first process is used for the current algorithm environment, and the second process is used to connect to the extended algorithm environment. A two-way communication is established between the first process and the second process, including read and write dual channels. The algorithm execution code and the built-in variables are transferred from the first process to the second process for execution via the write channel. During execution, execution process information and exception error information are read from the second process to the first process via the read channel and persisted to a local file for storage.

8. An online algorithm development device with dynamic management of the operating environment, characterized in that, include: A memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the online algorithm development method for dynamic management of the runtime environment as described in any one of claims 1 to 6.

9. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps in the online algorithm development method for dynamic management of the runtime environment as described in any one of claims 1 to 6.

Images