Algorithm verification system based on Notebook file

Abstract

The embodiment of the invention relates to an algorithm verification system based on a Notebook file. The system comprises a client, a management server, an algorithm resource library, a data set resource library, a mirror image resource library and a computing node cluster. The management server is respectively connected with the client, the algorithm resource library, the data set resource library, the mirror image resource library and the computing node cluster; the computing node cluster is further connected with the data set resource library and the mirror image resource library. According to the invention, the test consistency can be ensured, and the test efficiency can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an algorithm verification system based on Notebook files. Background Art

[0002] When testing / verifying software algorithms or model algorithms, the most ideal state is to regard the test object (software algorithm or model algorithm) as a black box that does not require adaptation. It is only necessary to prepare test data and send it to the black box and compare the black box output data with the expected data. This test efficiency is the highest. However, this ideal test efficiency is difficult to achieve in real tests. There are two main reasons: 1) Before the test, the host where the test object (software algorithm or model algorithm) is deployed needs to be installed by human operation. Affected by manual experience and work efficiency, this conventional processing method often has problems such as high error rate and long installation time, which leads to reduced test efficiency and difficulty in ensuring test consistency; 2) After obtaining the test results, it is necessary to import them into another test analysis tool by human operation to analyze whether they match the expected results. Affected by personnel work efficiency, this conventional processing method is also prone to problems such as untimely data import / export, which will also reduce test efficiency.

[0003] As we know, container (Docker) technology allows developers to package applications and their dependencies into a portable image file, which can be used to build automated container instances in any environment that supports Docker, and all built container environments are consistent. If the algorithm runtime environment can be installed based on container technology, it can not only eliminate the influence of human factors, improve deployment / testing efficiency, but also ensure test consistency. We also know that Notebook editors (such as Jupyter Notebook and Marimo Notebook) are intuitive interactive editors that can embed static text / rich text / images / audio / video through Markdown cells in the edited Notebook file, embed source code through code cells, and run the embedded source code. If the algorithm reasoning process source code, data set configuration code, algorithm verification process source code, and verification result analysis process source code can be integrated into a Notebook file based on the technical characteristics of the Notebook file, then the corresponding verification analysis report can be obtained by running the Notebook file once, thereby further improving the test efficiency. And how to introduce container technology and Notebook editor technology into the algorithm verification process is also the technical problem that the present invention needs to solve. Summary of the invention

[0004] The purpose of the present invention is to provide an algorithm verification system based on Notebook files in view of the defects of the prior art. The system includes: a client, a management server, an algorithm resource library, a data set resource library, an image resource library and a computing node cluster; the algorithm files stored in the algorithm resource library are all Notebook files, and the algorithm reasoning process source code, the data set configuration code, the algorithm verification process source code and the verification result analysis process source code are integrated into one algorithm file in accordance with the established format requirements, i.e., the algorithm Notebook file format requirements; the client is used to provide users with functions such as algorithm publishing, data set publishing and resource (algorithm, data set, image) information query through the management server. It can, and when the user determines to verify the specified algorithm based on the specified data set, send the verification request submitted by the user with the verification algorithm identifier and the verification data set identifier to the management server; the management server completes the automatic algorithm operation environment construction by creating a mirror container on the working node of the computing node cluster, and automatically obtains the target algorithm file to be verified through the verification algorithm identifier, and adaptively modifies the data set configuration information of the target algorithm file according to the data set parameters corresponding to the verification data set identifier based on the established format requirements, and automatically runs the modified target algorithm file in the algorithm operation environment, and sends the operation output file as the corresponding verification report, i.e., the first report file, to the client. The present invention can realize the automatic construction of the algorithm operation environment and ensure the consistency of the construction environment on the one hand, and realize the automatic linkage of algorithm verification and verification analysis on the other hand; the present invention can not only ensure the test consistency but also effectively improve the test efficiency.

[0005] To achieve the above-mentioned purpose, an embodiment of the present invention provides an algorithm verification system based on Notebook files, the system comprising: a client, a management server, an algorithm resource library, a data set resource library, an image resource library and a computing node cluster;

[0006] The management server is connected to the client, the algorithm resource library, the data set resource library, the mirror resource library and the computing node cluster respectively; the computing node cluster is also connected to the data set resource library and the mirror resource library respectively;

[0007] The client is used to forward the algorithm publishing file input by the user to the management server, and display the publishing algorithm identifier sent back by the management server to the current user; the algorithm publishing file includes the publishing algorithm name, the publishing algorithm file and the host configuration requirements of the algorithm running environment; the publishing algorithm file is a Notebook file, and the Notebook file format of the publishing algorithm file meets the preset algorithm Notebook file format requirements;

[0008] The client is also used to forward the data publishing file input by the user to the management server, and display the publishing data set identifier sent back by the management server to the current user; the data publishing file includes a publishing data set file name and a publishing data set file; the data publishing file is composed of one or more first verification-label data pairs, each of which is composed of first verification input data and first label output data;

[0009] The client is also used to forward the first query request input by the user to the management server, and display the first query result sent back by the management server to the current user; the first query request includes a first query type and a first query identifier; the first query type includes an algorithm resource query, a data set resource query and an image resource query;

[0010] The client is also used to forward the first verification request input by the user to the management server; and when receiving the first report file sent by the management server, display the first report file to the current user; the first verification request includes a verification algorithm identifier and a verification data set identifier;

[0011] The management server is used for, upon receiving the algorithm publishing file, performing algorithm publishing processing according to the algorithm publishing file, the mirror resource library and the algorithm resource library to obtain the corresponding publishing algorithm identifier and send it back to the client;

[0012] The management server is also used to, upon receiving the data publishing file, perform data publishing processing according to the data publishing file and the data set resource library to obtain the corresponding published data set identifier and send it back to the client;

[0013] The management server is further configured to, upon receiving the first query request, perform resource query processing according to the first query request, the algorithm resource library, the data set resource library, and the mirror resource library to obtain the corresponding first query result and send it back to the client;

[0014] The management server is further configured to, upon receiving the first verification request, perform algorithm verification processing according to the first verification request, the algorithm resource library, the data set resource library, the image resource library, and the computing node cluster to obtain the corresponding first report file and send it to the client;

[0015] The algorithm resource library is used to store a plurality of first algorithm information records;

[0016] The data set resource library is used to store a plurality of first verification data sets;

[0017] The image resource library is used to store multiple first algorithm running environment images;

[0018] The computing node cluster includes a first master node and multiple first working nodes; the first master node is connected to each of the first working nodes; each of the first working nodes corresponds to a node host configuration information.

[0019] Preferably, each of the first algorithm information records corresponds to an algorithm resource; the first algorithm information record includes at least a first algorithm identifier, a first algorithm name, a first algorithm file and a first algorithm image identifier; the first image identifier is the image identifier of the algorithm operating environment image corresponding to the current algorithm resource; the first algorithm identifier is the unique identifier of the current algorithm resource; the first algorithm name is the name of the current algorithm resource; the first algorithm file is the Notebook file of the current algorithm resource, and the Notebook file format of the first algorithm file meets the Notebook file format requirement; the first image identifier is the image identifier of the first algorithm operating environment image corresponding to the current algorithm resource;

[0020] Each of the first verification data sets corresponds to a data set configuration information; the data set configuration information is also stored in the data set resource library; the data set configuration information includes a first data set identifier, a first data set file name and a first data set resource download address; the first data set identifier is the unique identifier of the current first verification data set; the first data set file name is the data file name of the current first verification data set; the first data set resource download address is the data file download address of the current first verification data set; the data format of each of the first verification data sets is composed of one or more of the first verification-label data pairs, and each of the first verification-label data pairs is composed of the first verification input data and the first label output data;

[0021] Each of the first algorithm operating environment images corresponds to a set of image configuration information; each of the image configuration information is also stored in the image resource library; the image configuration information includes a first image identifier, a first image file name, a first image resource download address and a first image host configuration requirement; the first image identifier is the unique identifier of the current first algorithm operating environment image; the first image name is the name of the current first algorithm operating environment image; the first image file name is the image file name of the current first algorithm operating environment image; the first image resource download address is the image file download address of the current first algorithm operating environment image; the first image host configuration requirement is the minimum host configuration requirement of the current first algorithm operating environment image;

[0022] The first algorithm runtime environment image is an installation package image of a Notebook runtime environment; the Notebook runtime environment includes at least a Jupyter Notebook runtime environment and a Marimo Notebook runtime environment; the Notebook image includes at least an operating system component installer, a Notebook server installer, a program code interpreter installer for a Notebook editor, an editor kernel installer for a Notebook editor, a Notebook environment dependency library and extension library installer, a Notebook editor installer, an SSH component installer, an environment variable configuration program, and a main installer or main installation batch file for executing all installation / configuration programs one by one in a specified order; wherein the program code interpreter installer and the editor kernel installer correspond to specific source code programming languages; when the code programming language is Python programming language, the program code interpreter installer and the editor kernel installer are corresponding Python interpreter installer and IPython kernel installer; the code programming language is C / C++ programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding C / C++ interpreter installation program and C / C++ kernel installation program; when the code programming language is Java programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Java interpreter installation program and IJava kernel installation program; when the code programming language is R programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding R interpreter installation program and IR kernel installation program; when the code programming language is Julia programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Julia interpreter installation program and IJulia kernel installation program; the first algorithm running environment image includes one or more groups of the program code interpreter installation programs and the editor kernel installation programs for different types of the source code programming languages.

[0023] Preferably, the algorithm Notebook file format requirements are:

[0024] A code cell is designated as a data set parameter setting cell, and the code of the data set parameter setting cell performs corresponding address reference declaration and file name reference declaration on the first data set resource download address and the first data set file name of the first verification data set used to verify the current algorithm; and a code cell is designated as a data set download cell, and the code of the data set download cell downloads the corresponding data set file as the corresponding current verification data set according to the currently referenced data file download address and data set file name; and based on one or more code cells, the algorithm reasoning process source code is loaded; and based on one or more code cells, the algorithm verification process source code is loaded; and based on one or more code cells, the verification result analysis process source code is loaded;

[0025] The data format of the current verification data set consists of one or more first verification-label data pairs, each of which consists of the first verification input data and the first label output data;

[0026] The main entry function of the algorithm reasoning process source code is recorded as the corresponding algorithm calling function, and the algorithm main interface function is used to perform algorithm calculation according to function input data and output corresponding algorithm calculation data;

[0027] The algorithm verification process source code is used to input the first verification input data of each of the first verification-label data pairs of the current verification data set into the algorithm calling function for processing and use the algorithm calculation data output by the function as the corresponding first measured output data, and each of the first measured output data and the corresponding first label output data form a corresponding first measured-label data pair, and all the obtained first measured-label data pairs form a corresponding first result data set;

[0028] The verification result analysis process source code is used to perform corresponding statistical data analysis and / or algorithm performance analysis according to the first result data set according to one or more types of statistical algorithms and / or performance evaluation algorithms to obtain corresponding one or more statistical analysis results and / or performance analysis results, and create corresponding visualization instance objects for each of the statistical analysis results and / or the performance analysis results, and display and output each of the visualization instance objects.

[0029] Preferably, the management server is specifically used for extracting the corresponding published algorithm name, the published algorithm file and the algorithm running environment host configuration requirements from the algorithm publishing file when the corresponding published algorithm identifier is sent back to the client after the algorithm publishing process is performed according to the algorithm publishing file, the image resource library and the algorithm resource library; and taking the first algorithm running environment image whose first image host configuration requirements in the image resource library meet the algorithm running environment host configuration requirements as the corresponding first matching image; and taking the first image identifier corresponding to the first matching image whose first image host configuration requirements are the lowest as the corresponding current image identifier; and adding a new first algorithm information record in the algorithm resource library as the corresponding current newly added record; and assigning a unique identifier to the current newly added record as the corresponding current algorithm identifier; and setting the first algorithm identifier, first algorithm name, first algorithm file and first algorithm image identifier of the current newly added record to the corresponding current algorithm identifier, the published algorithm name, the published algorithm file and the current image identifier; and sending the current algorithm identifier back to the client as the corresponding published algorithm identifier.

[0030] Preferably, the management server is specifically used for extracting the corresponding published data set file name and the published data set file from the data publishing file when the corresponding published data set identifier is sent back to the client after data publishing processing is performed according to the data publishing file and the data set resource library; and storing the published data set file in the data set resource library as a newly added first verification data set; and allocating a unique data set identifier and a data file download address for the newly added first verification data set as the corresponding first data set identifier and the first data set resource download address; and using the published data set file name as the corresponding first data set file name; and forming a corresponding data set configuration information composed of the obtained first data set identifier, the first data set file name and the first data set resource download address and storing it in the data set resource library; and sending the first data set identifier obtained this time as the corresponding published data set identifier back to the client.

[0031] Preferably, the management server is specifically configured to, when the resource query processing is performed according to the first query request, the algorithm resource library, the data set resource library, and the mirror resource library to obtain the corresponding first query result and send it back to the client:

[0032] Step 61, extracting the first query type corresponding to the first query request;

[0033] Step 62, identifying the first query type;

[0034] Step 63, if the first query type is an algorithm resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then the first algorithm identifier in the image resource library and the first image file name and the first image host configuration requirement of the image configuration information that match the first algorithm image identifier of each first algorithm information record in the algorithm resource library form a corresponding first algorithm image summary, and the first algorithm identifier, the first algorithm name and the first algorithm image summary corresponding to each first algorithm information record in the algorithm resource library form a corresponding first algorithm summary, and all the obtained first algorithm summaries form a corresponding first query result; if the first query identifier is not empty, then the first algorithm information record whose first algorithm identifier in the algorithm resource library matches the first query identifier is extracted as the corresponding current algorithm information record, and the first image file name and the first image host configuration requirement of the image configuration information whose first algorithm identifier in the image resource library matches the first algorithm image identifier of the current algorithm information record form a corresponding current algorithm image summary, and the current algorithm information record and the current algorithm image summary form a corresponding first query result;

[0035] Step 64: if the first query type is a data set resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then all the data set configuration information stored in the data set resource library constitute the corresponding first query result; if the first query identifier is not empty, then extract the data set configuration information in the algorithm resource library whose first data set identifier matches the first query identifier as the corresponding first query result;

[0036] Step 65: if the first query type is an image resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then all the image configuration information stored in the image resource library constitute the corresponding first query result; if the first query identifier is not empty, then extract the image configuration information in the image resource library that matches the first image identifier and the first query identifier as the corresponding first query result;

[0037] Step 66, and sending the obtained first query result back to the client.

[0038] Preferably, the management server is specifically used for, when the algorithm verification process is performed according to the first verification request, the algorithm resource library, the data set resource library, the image resource library and the computing node cluster to obtain the corresponding first report file and send it to the client:

[0039] Step 71, extracting the corresponding verification algorithm identifier and the verification data set identifier from the first verification request;

[0040] Step 72: taking the first algorithm information record in the algorithm resource library that matches the first algorithm identifier with the verification algorithm identifier as a corresponding matching algorithm record; and extracting the first algorithm file of the matching algorithm record as a corresponding first algorithm copy file; and extracting the first algorithm image identifier of the matching algorithm record as a corresponding matching image identifier;

[0041] Step 73, the first data set file name and the first data set resource download address of the data set configuration information whose first data set identifier matches the verification data set identifier in the data set resource library; and based on the first data set resource download address and the first data set file name, the parameter assignment part of the address reference declaration and the file name reference declaration in the data set parameter setting cell of the first algorithm copy file is set;

[0042] Step 74: extract the first image resource download address and the first image host configuration requirement of the image configuration information corresponding to the first image identifier and the matching image identifier in the image resource library as the corresponding first image download address and the first host configuration requirement to form a corresponding first image container request and send it to the first master node of the computing node cluster; and receive the first container connection parameter sent back by the first master node;

[0043] The first container connection parameters include the container IP address, SSH port, SSH login name and SSH login password;

[0044] Step 75: Use the Notebook container corresponding to the first container connection parameter as the corresponding first container; and perform an SSH channel connection with the first container according to the first container connection parameter to obtain the corresponding first SSH channel; and transfer the first algorithm copy file to the running environment of the first container through the first SSH channel to save it as the corresponding temporary Notebook file; and in the running environment of the first container, call the running instruction of the Notebook editor through the first SSH channel to run all code cells and all Markdown cells in the temporary Notebook file to obtain a Notebook file with the running results of all cells, which is recorded as the corresponding running result file; and retrieve the running result file to the local server through the first SSH channel; and when the transmission of the running result file is completed, delete the temporary Notebook file and the running result file through the first SSH channel in the running environment of the first container; and disconnect the first SSH channel;

[0045] Step 76, and send the first disconnection request carrying the first container connection parameters to the first master node; and receive and wait for the first disconnection feedback sent back by the first master node within a preset first waiting time; if the first disconnection feedback is not received within the first waiting time or the first disconnection feedback received within the first waiting time is a failure, the first disconnection request is sent to the first master node again until the first disconnection feedback received is a success;

[0046] Step 77: Send the operation result file as the corresponding first report file to the client.

[0047] Preferably, the first master node of the computing node cluster is used to extract the corresponding first image download address and the first host configuration requirement from the first image container request when receiving the first image container request; and select the first working node whose node host configuration information meets the first host configuration requirement and whose node working condition is in an idle state as the corresponding current node; and download the first algorithm operating environment image on the first image download address to the current node to create a corresponding image container to obtain the corresponding Notebook container, and run the Notebook container; and perform corresponding IP address allocation, SSH port setting, SSSH login name setting and SSH login password setting for the Notebook container to obtain the corresponding container IP address, the SSH port, the SSH login name and the SSH login password to form the corresponding first container connection parameters and send them back to the management server; and set the node working condition of the current node to an occupied state;

[0048] The first master node of the computing node cluster is also used to extract the corresponding first container connection parameter from the first disconnection application when receiving the first disconnection application; and use the first working node where the Notebook container corresponding to the first container connection parameter is located as the corresponding current node; and identify the node working condition of the current node; if the current node working condition is an idle state, set a corresponding first disconnection feedback as success; if the current node working condition is an occupied state, delete the Notebook container running on the current node, if the container deletion fails, set a corresponding first disconnection feedback as failure, if the container deletion is successful, further release the node resources occupied by the Notebook container on the current node, if the resource release fails, set a corresponding first disconnection feedback as failure, if the resource release is successful, set a corresponding first disconnection feedback as success; and send the obtained first disconnection feedback back to the management server; and set the node working condition of the current node to an idle state.

[0049] An embodiment of the present invention provides an algorithm verification system based on Notebook files, the system comprising: a client, a management server, an algorithm resource library, a data set resource library, an image resource library and a computing node cluster; the algorithm files stored in the algorithm resource library are all Notebook files, and the algorithm reasoning process source code, the data set configuration code, the algorithm verification process source code and the verification result analysis process source code are integrated into one algorithm file according to the established format requirements, i.e., the algorithm Notebook file format requirements; the client is used to provide the user with functions such as algorithm publishing, data set publishing and resource (algorithm, data set, image) information query through the management server, and when the user determines to verify the specified algorithm based on the specified data set, the verification request submitted by the user with the verification algorithm identifier and the verification data set identifier is sent to the management server; the management server completes the automatic algorithm running environment construction by creating an image container on the working node of the computing node cluster, and automatically obtains the target algorithm file to be verified through the verification algorithm identifier, and adaptively modifies the data set configuration information of the target algorithm file according to the data set parameters corresponding to the verification data set identifier based on the established format requirements, and automatically runs the modified target algorithm file in the algorithm running environment, and sends the running output file as the corresponding verification report, i.e., the first report file, to the client. The embodiments of the present invention can realize the automatic construction of the algorithm running environment and ensure the consistency of the construction environment on the one hand, and can realize the automatic linkage of algorithm verification and verification analysis on the other hand; through the embodiments of the present invention, not only the test consistency is ensured, but also the test efficiency is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A module structure diagram of a Notebook file-based algorithm verification system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] like Figure 1As shown in the module structure diagram of an algorithm verification system based on Notebook files provided by an embodiment of the present invention, an algorithm verification system based on Notebook files provided by an embodiment of the present invention mainly includes: client 1, management server 2, algorithm resource library 3, data set resource library 4, mirror resource library 5 and computing node cluster 6. The connection relationship of each module is as follows: the management server 2 is connected to the client 1, algorithm resource library 3, data set resource library 4, mirror resource library 5 and computing node cluster 6 respectively; the computing node cluster 6 is also connected to the data set resource library 4 and the mirror resource library 5 respectively.

[0053] (I) Client 1:

[0054] The client 1 in the embodiment of the present invention is a web client.

[0055] The client 1 is used to forward the algorithm publishing file input by the user to the management server 2, and display the publishing algorithm identifier sent back by the management server 2 to the current user.

[0056] Here, the algorithm publishing file of the embodiment of the present invention includes a publishing algorithm name, a publishing algorithm file, and an algorithm running environment host configuration requirement; wherein the publishing algorithm file is a Notebook file, and the Notebook file format of the publishing algorithm file meets the preset algorithm Notebook file format requirement; the algorithm running environment host configuration requirement includes at least the total number of host CPU cores, host memory, whether a GPU is required, the required GPU video memory capacity, host storage capacity and other configuration information;

[0057] Here, the algorithm Notebook file format of the embodiment of the present invention is required to include at least the following contents:

[0058] 1) designating a code cell as a data set parameter setting cell, and using the code of the data set parameter setting cell to perform corresponding address reference declaration and file name reference declaration on a first data set resource download address and a first data set file name of a first verification data set used to verify the current algorithm;

[0059] 2) Designate a code cell as a data set download cell, and the code of the data set download cell downloads the corresponding data set file as the corresponding current verification data set according to the currently referenced data file download address and data set file name;

[0060] The data format of the current verification data set here is consistent with the data format of the published data set file submitted by the user below and the data format of each first verification data set in the data set resource library 4, that is, the current verification data set consists of one or more first verification-label data pairs, and each first verification-label data pair consists of first verification input data and first label output data;

[0061] 3) loading the algorithm reasoning process source code based on one or more code cells;

[0062] The main entry function of the algorithm reasoning process source code here is recorded as the corresponding algorithm calling function. The algorithm main interface function is used to perform algorithm calculation according to the function input data and output the corresponding algorithm calculation data;

[0063] 4) loading the algorithm verification process source code based on one or more code cells;

[0064] The algorithm verification process source code here is used to input the first verification input data of each first verification-label data pair of the current verification data set into the algorithm calling function for processing and use the algorithm calculation data output by the function as the corresponding first measured output data, and each first measured output data and the corresponding first label output data form a corresponding first measured-label data pair, and all the obtained first measured-label data pairs form a corresponding first result data set;

[0065] 5) loading the verification result analysis process source code based on one or more code cells;

[0066] The verification result analysis process source code here is used to perform corresponding statistical data analysis and / or algorithm performance analysis according to one or more types of statistical algorithms and / or performance evaluation algorithms based on the first result data set to obtain corresponding one or more statistical analysis results and / or performance analysis results, and create corresponding visualization instance objects for each statistical analysis result and / or performance analysis result, and display and output each visualization instance object.

[0067] It is not difficult to see from the above algorithm Notebook file format requirements that each time verification is performed based on a different data set, it is only necessary to adaptively update the address reference declaration and file name reference declaration parameters of the data set parameter setting cell; after standardizing the built-in code of the data set parameter setting cell, updating its internal reference declaration parameters can be used as an automated update process.

[0068] The client 1 is also used to forward the data publishing file input by the user to the management server 2, and display the publishing data set identifier sent back by the management server 2 to the current user.

[0069] Here, the data publishing file of an embodiment of the present invention includes a published data set file name and a published data set file; the data format of the data publishing file is: the data publishing file is composed of one or more first verification-label data pairs, and each first verification-label data pair is composed of first verification input data and first label output data.

[0070] The client 1 is further configured to forward the first query request input by the user to the management server 2 , and display the first query result sent back by the management server 2 to the current user.

[0071] Here, the first query request of the embodiment of the present invention includes a first query type and a first query identifier; wherein the first query type includes an algorithm resource query, a data set resource query, and an image resource query.

[0072] The client 1 is also used to forward the first verification request input by the user to the management server 2; and when receiving the first report file sent by the management server 2, display the first report file to the current user.

[0073] Here, the first verification request in the embodiment of the present invention includes a verification algorithm identifier and a verification data set identifier.

[0074] (II) Management Server 2:

[0075] The management server 2 of the embodiment of the present invention is a service module, device, terminal, equipment, server / server / service interface / microservice, system or platform for processing the service application of the client 1. The management server 2 of the embodiment of the present invention is a remote management terminal of the client 1, and can connect to multiple clients 1 at the same time.

[0076] The management server 2 is used for, when receiving the algorithm release file, performing algorithm release processing according to the algorithm release file, the mirror resource library 5 and the algorithm resource library 3 to obtain the corresponding release algorithm identifier and send it back to the client 1.

[0077] The management server 2 is also used to, when receiving a data publishing file, perform data publishing processing according to the data publishing file and the data set resource library 4 to obtain a corresponding publishing data set identifier and send it back to the client 1.

[0078] The management server 2 is also used for, when receiving the first query request, performing resource query processing according to the first query request, the algorithm resource library 3, the data set resource library 4 and the image resource library 5 to obtain the corresponding first query result and send it back to the client 1.

[0079] The management server 2 is also used to perform algorithm verification processing according to the first verification request, algorithm resource library 3, data set resource library 4, image resource library 5 and computing node cluster 6 when receiving the first verification request, and obtain the corresponding first report file and send it to the client 1.

[0080] In a specific implementation of an embodiment of the present invention, the management server 2 is specifically used to extract the corresponding publishing algorithm name, publishing algorithm file and algorithm running environment host configuration requirements from the algorithm publishing file when the corresponding publishing algorithm identifier is sent back to the client 1 after performing algorithm publishing processing according to the algorithm publishing file, the image resource library 5 and the algorithm resource library 3; and use the first algorithm running environment image whose first image host configuration requirements in the image resource library 5 meet the algorithm running environment host configuration requirements as the corresponding first matching image; and use the first image identifier corresponding to the first matching image with the lowest first image host configuration requirement as the corresponding current image identifier; and add a first algorithm information record in the algorithm resource library 3 as the corresponding current newly added record; and assign a unique identifier to the current newly added record as the corresponding current algorithm identifier; and set the first algorithm identifier, first algorithm name, first algorithm file and first algorithm image identifier of the current newly added record as the corresponding current algorithm identifier, publishing algorithm name, publishing algorithm file and current image identifier; and send the current algorithm identifier back to the client 1 as the corresponding publishing algorithm identifier.

[0081] In another specific implementation of the embodiment of the present invention, the management server 2 is specifically used to extract the corresponding publishing dataset file name and publishing dataset file from the data publishing file when the corresponding publishing dataset identifier is obtained by performing data publishing processing according to the data publishing file and the dataset resource library 4 and sending it back to the client 1; and store the publishing dataset file in the dataset resource library 4 as a newly added first verification dataset; and assign a unique dataset identifier and a data file download address to the newly added first verification dataset as the corresponding first dataset identifier and first dataset resource download address; and use the publishing dataset file name as the corresponding first dataset file name; and a corresponding dataset configuration information composed of the obtained first dataset identifier, first dataset file name and first dataset resource download address is stored in the dataset resource library 4; and the first dataset identifier obtained this time is sent back to the client 1 as the corresponding publishing dataset identifier.

[0082] In another specific implementation of the embodiment of the present invention, the management server 2 is specifically used for performing resource query processing according to the first query request, the algorithm resource library 3, the data set resource library 4 and the image resource library 5 to obtain the corresponding first query result and send it back to the client 1:

[0083] Step A1, extracting a corresponding first query type from a first query request;

[0084] Step A2, identifying the first query type;

[0085] Step A3, if the first query type is an algorithm resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then the first image file name and the first image host configuration requirement of the image configuration information whose first algorithm identifier in the image resource library 5 matches the first algorithm image identifier of each first algorithm information record in the algorithm resource library 3 form a corresponding first algorithm image summary, and the first algorithm identifier, the first algorithm name and the first algorithm image summary corresponding to each first algorithm information record in the algorithm resource library 3 form a corresponding first algorithm summary, and all the obtained first algorithm summaries form a corresponding first query result; if the first query identifier is not empty, then the first algorithm information record whose first algorithm identifier in the algorithm resource library 3 matches the first query identifier is extracted as the corresponding current algorithm information record, and the first image file name and the first image host configuration requirement of the image configuration information whose first algorithm identifier in the image resource library 5 matches the first algorithm image identifier of the current algorithm information record form a corresponding current algorithm image summary, and the current algorithm information record and the current algorithm image summary form a corresponding first query result;

[0086] Step A4, if the first query type is a data set resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then all the data set configuration information stored in the data set resource library 4 constitutes the corresponding first query result; if the first query identifier is not empty, then extract the data set configuration information whose first data set identifier in the algorithm resource library 3 matches the first query identifier as the corresponding first query result;

[0087] Step A5, if the first query type is an image resource query, then identify whether the first query identifier is empty; if the first query identifier is empty, then all the image configuration information stored in the image resource library 5 constitutes the corresponding first query result; if the first query identifier is not empty, then extract the image configuration information whose first image identifier in the image resource library 5 matches the first query identifier as the corresponding first query result;

[0088] Step A6, and send the obtained first query result back to client 1.

[0089] In another specific implementation of the embodiment of the present invention, the management server 2 is specifically used for performing algorithm verification processing according to the first verification request, the algorithm resource library 3, the data set resource library 4, the image resource library 5 and the computing node cluster 6 to obtain a corresponding first report file and send it to the client 1:

[0090] Step B1, extracting a corresponding verification algorithm identifier and a verification data set identifier from the first verification request;

[0091] Step B2, taking the first algorithm information record whose first algorithm identifier in the algorithm resource library 3 matches the verification algorithm identifier as the corresponding matching algorithm record; and extracting the first algorithm file of the matching algorithm record as the corresponding first algorithm copy file; and extracting the first algorithm mirror identifier of the matching algorithm record as the corresponding matching mirror identifier;

[0092] Step B3, the first data set file name and the first data set resource download address of the data set configuration information whose first data set identifier matches the verification data set identifier in the data set resource library 4; and the parameter assignment part of the address reference declaration and the file name reference declaration in the data set parameter setting cell of the first algorithm copy file is set based on the first data set resource download address and the first data set file name;

[0093] Step B4, extracting the first image resource download address and the first image host configuration requirement of the image configuration information corresponding to the first image identifier and the matching image identifier in the image resource library 5 as the corresponding first image download address and the first host configuration requirement to form a corresponding first image container request and sending it to the first master node 61 of the computing node cluster 6; and receiving the first container connection parameters sent back by the first master node 61;

[0094] The first container connection parameters include the container IP address, SSH port, SSH login name, and SSH login password;

[0095] Step B5, taking the Notebook container corresponding to the first container connection parameter as the corresponding first container; and performing an SSH channel connection with the first container according to the first container connection parameter to obtain the corresponding first SSH channel; and transferring the first algorithm copy file to the running environment of the first container through the first SSH channel to save it as the corresponding temporary Notebook file; and in the running environment of the first container, calling the running instruction of the Notebook editor through the first SSH channel to run all code cells and all Markdown cells in the temporary Notebook file to obtain a Notebook file with the running results of all cells as the corresponding running result file; and retrieving the running result file to the local server through the first SSH channel; and when the running result file transmission is completed, deleting the temporary Notebook file and the running result file through the first SSH channel in the running environment of the first container; and disconnecting the first SSH channel;

[0096] Step B6, and send the first disconnection request carrying the first container connection parameters to the first master node 61; and receive and wait for the first disconnection feedback sent back by the first master node 61 within a preset first waiting time; if the first disconnection feedback is not received within the first waiting time or the first disconnection feedback received within the first waiting time is a failure, the first disconnection request is sent to the first master node 61 again until the first disconnection feedback received is a success;

[0097] Step B7: sending the operation result file as the corresponding first report file to the client 1.

[0098] (III) Algorithm Resource Library 3:

[0099] The algorithm resource library 3 of the embodiment of the present invention can be implemented based on any type of database, can also be implemented based on a combination of multiple types of databases, and can also be implemented based on a comprehensive data processing system that combines one or more types of storage systems with databases.

[0100] The algorithm resource library 3 is used to store multiple first algorithm information records. Each first algorithm information record corresponds to an algorithm resource; the first algorithm information record includes at least a first algorithm identifier, a first algorithm name, a first algorithm file, and a first algorithm image identifier; the first image identifier is the image identifier of the algorithm operating environment image corresponding to the current algorithm resource; the first algorithm identifier is the unique identifier of the current algorithm resource; the first algorithm name is the name of the current algorithm resource; the first algorithm file is the Notebook file of the current algorithm resource, and the Notebook file format of the first algorithm file meets the Notebook file format requirements; the first image identifier is the image identifier of the first algorithm operating environment image corresponding to the current algorithm resource.

[0101] (IV) Dataset Resource Library 4:

[0102] The data set resource library 4 of the embodiment of the present invention can be implemented based on any type of database, can also be implemented based on a combination of multiple types of databases, and can also be implemented based on a comprehensive data processing system that combines one or more types of storage systems with databases.

[0103] The data set resource library 4 is used to store multiple first verification data sets. Each first verification data set corresponds to a data set configuration information; the data set configuration information is also stored in the data set resource library 4; the data set configuration information includes a first data set identifier, a first data set file name, and a first data set resource download address; the first data set identifier is the unique identifier of the current first verification data set; the first data set file name is the data file name of the current first verification data set; the first data set resource download address is the data file download address of the current first verification data set; the data format of each first verification data set consists of one or more first verification-label data pairs, and each first verification-label data pair consists of first verification input data and first label output data.

[0104] (V) Image Resource Library 5:

[0105] The mirror resource library 5 of the embodiment of the present invention can be implemented based on any type of database, can also be implemented based on a combination of multiple types of databases, and can also be implemented based on a comprehensive data processing system that combines one or more types of storage systems with databases.

[0106] The image resource library 5 is used to store multiple first algorithm operating environment images. Each first algorithm operating environment image corresponds to a set of image configuration information; each image configuration information is also stored in the image resource library 5; the image configuration information includes a first image identifier, a first image file name, a first image resource download address and a first image host configuration requirement; the first image identifier is the unique identifier of the current first algorithm operating environment image; the first image name is the name of the current first algorithm operating environment image; the first image file name is the image file name of the current first algorithm operating environment image; the first image resource download address is the image file download address of the current first algorithm operating environment image; the first image host configuration requirement is the minimum host configuration requirement of the current first algorithm operating environment image, and the configuration items of the first image host configuration requirement include at least the total number of host CPU cores, host memory, whether a GPU is required, the required GPU video memory capacity, host storage capacity and other configuration information.

[0107] It should be noted that the first algorithm operating environment image of the embodiment of the present invention is an installation package image of a Notebook operating environment; the Notebook operating environment includes at least the Jupyter Notebook operating environment and the Marimo Notebook operating environment; the Notebook image includes at least an operating system component installer, a Notebook server installer, a program code interpreter installer of the Notebook editor, an editor kernel installer of the Notebook editor, a Notebook environment dependency library and extension library installer, a Notebook editor installer, an SSH component installer, an environment variable configuration program, and a main installer or a main installation batch file for executing all installation / configuration programs one by one in a specified order;

[0108] Among them, the program code interpreter installation program and the editor kernel installation program correspond to the specific source code programming language; when the code programming language is Python programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Python interpreter installation program and IPython kernel installation program; when the code programming language is C / C++ programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding C / C++ interpreter installation program and C / C++ kernel installation program; when the code programming language is Java programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Java interpreter installation program and IJava kernel installation program; when the code programming language is R programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding R interpreter installation program and IR kernel installation program; when the code programming language is Julia programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Julia interpreter installation program and IJulia kernel installation program;

[0109] The first algorithm runtime environment image of the embodiment of the present invention includes one or more sets of program code interpreter installation programs and editor kernel installation programs for different types of source code programming languages.

[0110] (VI) Computing Node Cluster 6:

[0111] The computing node cluster 6 of the embodiment of the present invention may be composed of multiple nodes deployed in one cluster, or may be composed of multiple sub-clusters deployed in multiple regions.

[0112] The computing node cluster 6 includes a first master node 61 and a plurality of first working nodes 62; the first master node 61 is connected to each first working node 62; each first working node 62 corresponds to a node host configuration information. The configuration items of the node host configuration information at least include the total number of host CPU cores, host memory, whether a GPU is required, the required GPU memory capacity, host storage capacity and other configuration information.

[0113] The first master node 61 of the computing node cluster 6 is used to extract the corresponding first image download address and the first host configuration requirement from the first image container request when receiving the first image container request; and select a first working node 62 whose node host configuration information meets the first host configuration requirement and whose node working condition is idle as the corresponding current node; and download the first algorithm running environment image on the first image download address to the current node to create the corresponding image container to obtain the corresponding Notebook container, and run the Notebook container; and perform corresponding IP address allocation, SSH port setting, SSSH login name setting and SSH login password setting for the Notebook container to obtain the corresponding container IP address, SSH port, SSH login name and SSH login password to form the corresponding first container connection parameters and send them back to the management server 2; and set the node working condition of the current node to the occupied state.

[0114] The first master node 61 of the computing node cluster 6 is also used to extract the corresponding first container connection parameters from the first disconnection application when receiving the first disconnection application; and use the first working node 62 where the Notebook container corresponding to the first container connection parameters is located as the corresponding current node; and identify the node working condition of the current node; if the current node working condition is an idle state, set a corresponding first disconnection feedback as success; if the current node working condition is an occupied state, delete the Notebook container running on the current node, if the container deletion fails, set a corresponding first disconnection feedback as failure, if the container deletion is successful, further release the node resources occupied by the Notebook container on the current node, if the resource release fails, set a corresponding first disconnection feedback as failure, if the resource release is successful, set a corresponding first disconnection feedback as success; and send the obtained first disconnection feedback back to the management server 2; and set the node working condition of the current node to an idle state.

[0115] It should be noted that the management software used by the computing node cluster 6 of the embodiment of the present invention when performing resource management on all first working nodes 62 through the first master node 61 internally includes at least Kubernetes and Zabbix, and the corresponding operating system, communication components, network components and the master / slave server of the management software are installed on the first master node 61 and each first working node 62 based on the Kubernetes or Zabbix installation requirements. The embodiment of the present invention does not limit the technical content related to the internal management of the computing node cluster 6, and will not be further elaborated here.

[0116] It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in software form; they can also all be implemented in hardware form; some modules can also be implemented in software form, and some modules can be implemented in hardware form. For example, the management service end can be a separately established processing device, equipment or server, or it can be integrated in a module of the above system for implementation. In addition, it can also be stored in the memory of the above system in the form of program code, and called and executed by a processing module of the above system. The module function determined above. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing module described here can be an integrated circuit with signal processing capabilities. In the implementation process, the processing steps of each module of the aforementioned system can be completed by the integrated logic circuit of the hardware in the processing module or the instructions in the form of software.

[0117] For example, the modules of the above system may be one or more integrated circuits configured to implement the aforementioned system, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module of the above system is implemented in the form of a processing module scheduling program code, the processing module may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0118] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0119] An embodiment of the present invention provides an algorithm verification system based on Notebook files, the system comprising: a client, a management server, an algorithm resource library, a data set resource library, an image resource library and a computing node cluster; the algorithm files stored in the algorithm resource library are all Notebook files, and the algorithm reasoning process source code, the data set configuration code, the algorithm verification process source code and the verification result analysis process source code are integrated into one algorithm file according to the established format requirements, i.e., the algorithm Notebook file format requirements; the client is used to provide the user with functions such as algorithm publishing, data set publishing and resource (algorithm, data set, image) information query through the management server, and when the user determines to verify the specified algorithm based on the specified data set, the verification request submitted by the user with the verification algorithm identifier and the verification data set identifier is sent to the management server; the management server completes the automatic algorithm running environment construction by creating an image container on the working node of the computing node cluster, and automatically obtains the target algorithm file to be verified through the verification algorithm identifier, and adaptively modifies the data set configuration information of the target algorithm file according to the data set parameters corresponding to the verification data set identifier based on the established format requirements, and automatically runs the modified target algorithm file in the algorithm running environment, and sends the running output file as the corresponding verification report, i.e., the first report file, to the client. The embodiments of the present invention can realize the automatic construction of the algorithm running environment and ensure the consistency of the construction environment on the one hand, and can realize the automatic linkage of algorithm verification and verification analysis on the other hand; through the embodiments of the present invention, not only the test consistency is ensured, but also the test efficiency is effectively improved.

[0120] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0121] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0122] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An algorithm verification system based on Notebook files, characterized in that: The system includes: a client, a management server, an algorithm resource library, a data set resource library, a mirror resource library and a computing node cluster; The management server is connected to the client, the algorithm resource library, the data set resource library, the mirror resource library and the computing node cluster respectively; the computing node cluster is also connected to the data set resource library and the mirror resource library respectively; The client is used to forward the algorithm publishing file input by the user to the management server, and display the publishing algorithm identifier sent back by the management server to the current user; the algorithm publishing file includes the publishing algorithm name, the publishing algorithm file and the host configuration requirements of the algorithm running environment; the publishing algorithm file is a Notebook file, and the Notebook file format of the publishing algorithm file meets the preset algorithm Notebook file format requirements; The client is also used to forward the data publishing file input by the user to the management server, and display the publishing data set identifier sent back by the management server to the current user; the data publishing file includes a publishing data set file name and a publishing data set file; the data publishing file is composed of one or more first verification-label data pairs, each of which is composed of first verification input data and first label output data; The client is also used to forward the first query request input by the user to the management server, and display the first query result sent back by the management server to the current user; the first query request includes a first query type and a first query identifier; the first query type includes an algorithm resource query, a data set resource query, and an image resource query; The client is also used to forward the first verification request input by the user to the management server; and when receiving the first report file sent by the management server, display the first report file to the current user; the first verification request includes a verification algorithm identifier and a verification data set identifier; The management server is used for, upon receiving the algorithm publishing file, performing algorithm publishing processing according to the algorithm publishing file, the mirror resource library and the algorithm resource library to obtain the corresponding publishing algorithm identifier and send it back to the client; The management server is also used to, upon receiving the data publishing file, perform data publishing processing according to the data publishing file and the data set resource library to obtain the corresponding published data set identifier and send it back to the client; The management server is further configured to, upon receiving the first query request, perform resource query processing according to the first query request, the algorithm resource library, the data set resource library, and the mirror resource library to obtain the corresponding first query result and send it back to the client; The management server is further configured to, upon receiving the first verification request, perform algorithm verification processing according to the first verification request, the algorithm resource library, the data set resource library, the image resource library, and the computing node cluster to obtain the corresponding first report file and send it to the client; The algorithm resource library is used to store a plurality of first algorithm information records; The data set resource library is used to store a plurality of first verification data sets; The image resource library is used to store multiple first algorithm running environment images; The computing node cluster includes a first master node and multiple first working nodes; the first master node is connected to each of the first working nodes; each of the first working nodes corresponds to a node host configuration information.

2. The algorithm verification system based on Notebook file according to claim 1, characterized in that: Each of the first algorithm information records corresponds to an algorithm resource; the first algorithm information record includes at least a first algorithm identifier, a first algorithm name, a first algorithm file and a first algorithm image identifier; the first image identifier is an image identifier of an algorithm operating environment image corresponding to the current algorithm resource; the first algorithm identifier is a unique identifier of the current algorithm resource; the first algorithm name is a name of the current algorithm resource; the first algorithm file is a Notebook file of the current algorithm resource, and the Notebook file format of the first algorithm file meets the Notebook file format requirement; The first image identifier is an image identifier of a first algorithm operating environment image corresponding to the current algorithm resource; Each of the first verification data sets corresponds to a data set configuration information; the data set configuration information is also stored in the data set resource library; the data set configuration information includes a first data set identifier, a first data set file name and a first data set resource download address; The first data set identifier is a unique identifier of the current first verification data set; The first data set file name is the data file name of the current first verification data set; The first data set resource download address is the data file download address of the current first verification data set; the data format of each first verification data set consists of one or more first verification-label data pairs, and each first verification-label data pair consists of the first verification input data and the first label output data; Each of the first algorithm running environment images corresponds to a set of image configuration information; each of the image configuration information is also stored in the image resource library; the image configuration information includes a first image identifier, a first image file name, a first image resource download address and a first image host configuration requirement; the first image identifier is the unique identifier of the current first algorithm running environment image; the first image name is the name of the current first algorithm running environment image; the first image file name is the image file name of the current first algorithm running environment image; The first image resource download address is the image file download address of the current first algorithm running environment image; the first image host configuration requirement is the minimum host configuration requirement of the current first algorithm running environment image; The first algorithm runtime environment image is an installation package image of a Notebook runtime environment; the Notebook runtime environment includes at least a Jupyter Notebook runtime environment and a Marimo Notebook runtime environment; the Notebook image includes at least an operating system component installer, a Notebook server installer, a program code interpreter installer for a Notebook editor, an editor kernel installer for a Notebook editor, a Notebook environment dependency library and extension library installer, a Notebook editor installer, an SSH component installer, an environment variable configuration program, and a main installer or main installation batch file for executing all installation / configuration programs one by one in a specified order; wherein the program code interpreter installer and the editor kernel installer correspond to specific source code programming languages; when the code programming language is Python programming language, the program code interpreter installer and the editor kernel installer are corresponding Python interpreter installer and IPython kernel installer; the code programming language is C / C++ programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding C / C++ interpreter installation program and C / C++ kernel installation program; when the code programming language is Java programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Java interpreter installation program and IJava kernel installation program; when the code programming language is R programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding R interpreter installation program and IR kernel installation program; when the code programming language is Julia programming language, the program code interpreter installation program and the editor kernel installation program are the corresponding Julia interpreter installation program and IJulia kernel installation program; the first algorithm running environment image includes one or more groups of the program code interpreter installation programs and the editor kernel installation programs for different types of the source code programming languages.

3. The algorithm verification system based on Notebook file according to claim 2, characterized in that: The algorithm Notebook file format requirements are: A code cell is designated as a data set parameter setting cell, and the code of the data set parameter setting cell performs corresponding address reference declaration and file name reference declaration on the first data set resource download address and the first data set file name of the first verification data set used to verify the current algorithm; And designate a code cell as a data set download cell, and the code of the data set download cell downloads the corresponding data set file as the corresponding current verification data set according to the currently referenced data file download address and data set file name; and loading the algorithm reasoning process source code based on one or more code cells; and loading the algorithm verification process source code based on one or more code cells; and loading the verification result analysis process source code based on one or more code cells; The data format of the current verification data set consists of one or more first verification-label data pairs, each of which consists of the first verification input data and the first label output data; The main entry function of the algorithm reasoning process source code is recorded as the corresponding algorithm calling function, and the algorithm main interface function is used to perform algorithm calculation according to function input data and output corresponding algorithm calculation data; The algorithm verification process source code is used to input the first verification input data of each of the first verification-label data pairs of the current verification data set into the algorithm calling function for processing and use the algorithm calculation data output by the function as the corresponding first measured output data, and each of the first measured output data and the corresponding first label output data form a corresponding first measured-label data pair, and all the obtained first measured-label data pairs form a corresponding first result data set; The verification result analysis process source code is used to perform corresponding statistical data analysis and / or algorithm performance analysis according to the first result data set according to one or more types of statistical algorithms and / or performance evaluation algorithms to obtain corresponding one or more statistical analysis results and / or performance analysis results, and create corresponding visualization instance objects for each of the statistical analysis results and / or the performance analysis results, and display and output each of the visualization instance objects.

4. The algorithm verification system based on Notebook file according to claim 2, characterized in that: The management server is specifically used for extracting the corresponding publishing algorithm name, the publishing algorithm file and the algorithm running environment host configuration requirements from the algorithm publishing file when the corresponding publishing algorithm identifier is obtained by performing algorithm publishing processing according to the algorithm publishing file, the image resource library and the algorithm resource library and sending it back to the client; and taking the first algorithm running environment image whose first image host configuration requirements in the image resource library meet the algorithm running environment host configuration requirements as the corresponding first matching image; And the first image identifier corresponding to the first matching image with the lowest configuration requirement of the first image host is used as the corresponding current image identifier; and a new first algorithm information record is added to the algorithm resource library as the corresponding current newly added record; and a unique identifier is assigned to the current newly added record as the corresponding current algorithm identifier; and the first algorithm identifier, first algorithm name, first algorithm file and first algorithm image identifier of the current newly added record are set to the corresponding current algorithm identifier, the published algorithm name, the published algorithm file and the current image identifier; and the current algorithm identifier is sent back to the client as the corresponding published algorithm identifier.

5. The algorithm verification system based on Notebook file according to claim 2, characterized in that: The management server is specifically used for extracting the corresponding published dataset file name and the published dataset file from the data publishing file when the corresponding published dataset identifier is obtained by performing data publishing processing according to the data publishing file and the dataset resource library and sending it back to the client; and storing the published data set file into the data set resource library as a newly added first verification data set; and assigning a unique data set identifier and a data file download address to the newly added first verification data set as the corresponding first data set identifier and the first data set resource download address; and taking the published data set file name as the corresponding first data set file name; The obtained first data set identifier, the first data set file name and the first data set resource download address form a corresponding data set configuration information and store it in the data set resource library; The first data set identifier obtained this time is sent back to the client as the corresponding published data set identifier.

6. The algorithm verification system based on Notebook file according to claim 2, characterized in that: The management server is specifically configured to, when the resource query processing is performed according to the first query request, the algorithm resource library, the data set resource library, and the mirror resource library to obtain the corresponding first query result and send it back to the client: Step 61, extracting the first query type corresponding to the first query request; Step 62, identifying the first query type; Step 63: if the first query type is an algorithm resource query, identifying whether the first query identifier is empty; If the first query identifier is empty, the first image file name and the first image host configuration requirement of the image configuration information whose first algorithm identifier in the image resource library matches the first algorithm image identifier of each first algorithm information record in the algorithm resource library are combined into a corresponding first algorithm image summary, and the first algorithm identifier, the first algorithm name and the first algorithm image summary corresponding to each first algorithm information record in the algorithm resource library are combined into a corresponding first algorithm summary, and all the obtained first algorithm summaries are combined into a corresponding first query result; if the first query identifier is not empty, the first algorithm information record whose first algorithm identifier in the algorithm resource library matches the first query identifier is extracted as the corresponding current algorithm information record, and the first image file name and the first image host configuration requirement of the image configuration information whose first algorithm identifier in the image resource library matches the first algorithm image identifier of the current algorithm information record are combined into a corresponding current algorithm image summary, and the current algorithm information record and the current algorithm image summary are combined into a corresponding first query result; Step 64: if the first query type is a data set resource query, identifying whether the first query identifier is empty; If the first query identifier is empty, all the data set configuration information stored in the data set resource library constitutes the corresponding first query result; If the first query identifier is not empty, extracting the data set configuration information in the algorithm resource library whose first data set identifier matches the first query identifier as the corresponding first query result; Step 65: If the first query type is a mirror resource query, identifying whether the first query identifier is empty; If the first query identifier is empty, all the image configuration information stored in the image resource library constitutes the corresponding first query result; If the first query identifier is not empty, extracting the image configuration information in the image resource library whose first image identifier matches the first query identifier as the corresponding first query result; Step 66, and sending the obtained first query result back to the client.

7. The algorithm verification system based on Notebook file according to claim 3, characterized in that: The management server is specifically configured to, when the algorithm verification process is performed according to the first verification request, the algorithm resource library, the data set resource library, the image resource library and the computing node cluster to obtain the corresponding first report file and send it to the client: Step 71, extracting the corresponding verification algorithm identifier and the verification data set identifier from the first verification request; Step 72: taking the first algorithm information record in the algorithm resource library that matches the first algorithm identifier with the verification algorithm identifier as a corresponding matching algorithm record; and extracting the first algorithm file of the matching algorithm record as a corresponding first algorithm copy file; and extracting the first algorithm image identifier of the matching algorithm record as a corresponding matching image identifier; Step 73, downloading the first dataset file name and the first dataset resource download address of the dataset configuration information whose first dataset identifier matches the verification dataset identifier in the dataset resource library; and setting the parameter assignment part of the address reference declaration and the file name reference declaration in the data set parameter setting cell of the first algorithm copy file based on the first data set resource download address and the first data set file name; Step 74: extract the first image resource download address and the first image host configuration requirement of the image configuration information corresponding to the first image identifier and the matching image identifier in the image resource library as the corresponding first image download address and the first host configuration requirement to form a corresponding first image container request and send it to the first master node of the computing node cluster; and receive the first container connection parameter sent back by the first master node; The first container connection parameters include the container IP address, SSH port, SSH login name and SSH login password; Step 75: Use the Notebook container corresponding to the first container connection parameter as the corresponding first container; and perform an SSH channel connection with the first container according to the first container connection parameter to obtain the corresponding first SSH channel; and transfer the first algorithm copy file to the running environment of the first container through the first SSH channel to save it as the corresponding temporary Notebook file; and in the running environment of the first container, call the running instruction of the Notebook editor through the first SSH channel to run all code cells and all Markdown cells in the temporary Notebook file to obtain a Notebook file with the running results of all cells, which is recorded as the corresponding running result file; and retrieve the running result file to the local server through the first SSH channel; and when the transmission of the running result file is completed, delete the temporary Notebook file and the running result file through the first SSH channel in the running environment of the first container; and disconnect the first SSH channel; Step 76, and send the first disconnection request carrying the first container connection parameters to the first master node; and receive and wait for the first disconnection feedback sent back by the first master node within a preset first waiting time; if the first disconnection feedback is not received within the first waiting time or the first disconnection feedback received within the first waiting time is a failure, the first disconnection request is sent to the first master node again until the first disconnection feedback received is a success; Step 77: Send the operation result file as the corresponding first report file to the client.

8. The algorithm verification system based on Notebook file according to claim 7, characterized in that: The first master node of the computing node cluster is used to extract the corresponding first image download address and the first host configuration requirement from the first image container request when receiving the first image container request; and select the first working node whose node host configuration information meets the first host configuration requirement and whose node working condition is idle as the corresponding current node; and download the first algorithm running environment image on the first image download address to the current node to create a corresponding image container to obtain the corresponding Notebook container, and run the Notebook container; And perform corresponding IP address allocation, SSH port setting, SSH login name setting and SSH login password setting for the Notebook container to obtain the corresponding container IP address, the SSH port, the SSH login name and the SSH login password to form the corresponding first container connection parameters and send them back to the management server; and set the node working state of the current node to an occupied state; The first master node of the computing node cluster is also used to extract the corresponding first container connection parameter from the first disconnection application when receiving the first disconnection application; and use the first working node where the Notebook container corresponding to the first container connection parameter is located as the corresponding current node; and identify the node working condition of the current node; if the current node working condition is an idle state, set a corresponding first disconnection feedback as success; if the current node working condition is an occupied state, delete the Notebook container running on the current node, if the container deletion fails, set a corresponding first disconnection feedback as failure, if the container deletion is successful, further release the node resources occupied by the Notebook container on the current node, if the resource release fails, set a corresponding first disconnection feedback as failure, if the resource release is successful, set a corresponding first disconnection feedback as success; and send the obtained first disconnection feedback back to the management server; and set the node working condition of the current node to an idle state.

Images