Data competition automatic evaluation method and system

Abstract

The invention belongs to the technical field of Internet, and discloses a data competition automatic evaluation method and system, and the method comprises the steps: receiving an evaluation task, carrying out the editing and storage of evaluation configuration according to the requirements of a competition question, and starting an evaluation environment through a scheduling program; queue monitoring is automatically established, and corresponding files are downloaded; and performing evaluation processing by using the scheduling mirror image and the execution mirror image of the evaluation container to obtain an evaluation score. According to the method, distributed deployment is adopted, different game question evaluation environments are isolated, and evaluation environment pollution is avoided; according to the method, automatic capacity expansion and recovery can be realized, so that the disaster recovery capability of the evaluation program is guaranteed; according to the invention, the queue mechanism ensures the evaluation robustness, and high-concurrency tasks can be processed easily; the container is automatically destroyed after expiration, so that server resources are saved; the dynamic matching of the evaluation script and the result set submits the task, and the game question maintenance cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of Internet technology, and in particular, relates to a data competition automatic evaluation method and system. Background Art

[0002] In recent years, with the rapid development of the Internet and big data technology, the number of people engaged in related industries has also been increasing, and data competitions have been born. Data competition is a new term that has emerged in recent years. It refers to an open competition model in the field of data science that is based on real data sets and real business problems, organized by enterprises and various authoritative institutions, and gathers a large number of high-quality data scientists and enthusiasts in a short period of time, with high-end data algorithms or data solution innovations as the result-oriented. Simply put, a data competition is like a "mathematical modeling competition" in the field of data science.

[0003] Through data science competitions, enterprises, research institutions and governments can provide some data to motivate data science talents around the world to help them solve some practical problems. At the same time, they can also use data science competitions to identify various outstanding data science talents and encourage them to find breakthrough methods for a certain data field or application scenario.

[0004] During the competition, after users have achieved certain results, they need to score and store the results. In order to solve the above problems, relevant personnel have done some research on the evaluation process. The existing evaluation is a stand-alone evaluation environment based on the nodejs program. Task triggering is performed on the same independent server, and the results are stored after scoring.

[0005] In large-scale competition scenarios, when a single machine is deployed with high-frequency and high-concurrency tasks, the server CPU and memory usage will be too high, and there is a risk of downtime causing business blockage. This study uses cloud computing and big data related technologies to build a competition evaluation system based on queue services, which can provide a distributed cluster environment for online evaluation and scoring of submission results of large-scale competition systems; it can automatically judge the submitted files of the contestants in real time and return the corresponding scores according to the program use cases set in the competition questions; it can effectively solve abnormal situations such as excessive CPU and memory usage and program jams in single-machine deployment scenarios, so as to better provide services for the competition system. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a method and system for automatic evaluation of data competition.

[0007] The present invention is implemented in this way: a data competition automatic evaluation method comprises:

[0008] Receive the evaluation task, edit and save the evaluation configuration according to the requirements of the competition, and then start the evaluation environment through the scheduler;

[0009] After the evaluation environment is started, it automatically establishes a connection with the queue service and continuously listens for new tasks. When a new task is obtained, the task content is parsed. After the scheduling image completes the content parsing, it will download the files required for the evaluation. When all the files required for the evaluation are complete, the evaluation information is transmitted to the execution image through zmq for evaluation. After the execution image obtains the evaluation task information, it will call the evaluation script and the answer file according to a specific method to perform a scoring operation to obtain the submitted score. After the execution image completes the scoring operation, it will return the score to the scheduling image, and the scheduling image will write back the score based on the returned information.

[0010] Furthermore, after editing and saving the evaluation configuration according to the requirements of the competition, before starting the evaluation environment through the scheduling program, the following steps need to be performed:

[0011] Upload the evaluation script and result set files of the competition questions in sequence through the management backend, and save the corresponding storage path, evaluation environment configuration, number of evaluation environment copies, and evaluation deadline into the database.

[0012] Furthermore, the corresponding files include: evaluation scripts, standard files and submission files.

[0013] Further, the use of the scheduling image and the execution image of the evaluation container to perform evaluation processing respectively includes:

[0014] The scheduling image is used to download and save the task file, and the original evaluation information is parsed and then processed; the relevant information is shared with the execution image under the same container through the TCP protocol. After receiving the task information, the execution image scores the evaluation task and returns the result to the scheduling image after processing; the scheduling image returns the information to the scheduler through the queue for storage.

[0015] Further, the use of the scheduling image and the execution image of the evaluation container to perform evaluation processing respectively includes the following steps:

[0016] (1) Use the scheduling image to establish a connection with the queue program and register the queue monitor. At the same time, the scheduling image downloads the saved evaluation script and result set file to the shared directory while ensuring that the connection is normal;

[0017] (2) The execution image and the scheduling image share information through the TCP protocol.

[0018] Furthermore, the data competition automated evaluation method further includes:

[0019] Submit the evaluation file through the website. After successful submission, the scheduler is triggered. When the scheduler starts, a queue connection is created. After receiving the evaluation task, the information is sorted and delivered to the message queue.

[0020] After the scheduling image listens to the new task through the queue, it will download the corresponding file, save the file to the specified directory and organize the evaluation information; after the scheduling image successfully saves the evaluation file, it will share the evaluation information to the execution image through zmq;

[0021] After the execution image obtains the evaluation information, it parses the content and scores the evaluation file through the evaluation script; the execution image shares the result to the scheduling image through zmq; the scheduling image obtains the information and sends the result back to the scheduler;

[0022] After receiving the information, the scheduler will write it back to the website through an encrypted interface.

[0023] Another object of the present invention is to provide a data contest automated evaluation system for implementing the data contest automated evaluation method, the data contest automated evaluation system comprising:

[0024] The storage module is used to store relevant documents and data through the scheduling program;

[0025] Evaluation configuration module, used to edit the evaluation configuration according to the requirements of the competition topic;

[0026] The evaluation environment startup module is used to start the evaluation environment through the scheduler;

[0027] Task execution module, used to execute evaluation tasks;

[0028] The anomaly detection module is used to detect task anomalies and automatically recover when an anomaly occurs in the task.

[0029] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:

[0030] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the technical solutions to be protected by the present invention and the results and data during the research and development process are closely combined to analyze in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows:

[0031] The present invention isolates the evaluation environment through distributed deployment and configures different images for different competition questions, allowing independent expansion or modification of competition question configuration to meet evaluation requirements in different scenarios;

[0032] The present invention has an automatic recovery mechanism for task anomaly detection, which will not affect the entire system and ensure that the evaluation task proceeds smoothly;

[0033] The present invention enables high-concurrency tasks to be cached and processed in batches through queue distribution, which helps to control and optimize the speed of data flow through the system, solve the problem of inconsistent processing speed of production messages and consumption messages, and reduce the pressure on servers and interfaces;

[0034] The present invention realizes automatic destruction of containers in the evaluation environment by configuring evaluation information, and provides a capability of automatically releasing resources;

[0035] The present invention upgrades the granularity of the evaluation configuration, provides resources of different magnitudes for different competition questions, and realizes server resource throttling.

[0036] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0037] The present invention customizes the evaluation requirements for different competition topics and isolates the evaluation environments of different competition topics to avoid pollution of the evaluation environment. At the same time, the evaluation environment has automatic expansion and recovery, so that the disaster recovery capability of the evaluation program is guaranteed. At the same time, it has automatic destruction capability, which saves server resources and provides a good operating environment for the development of competition topics.

[0038] Third, as auxiliary evidence of the creativity of the claims of the present invention, it is also reflected in the expected benefits and commercial value after the transformation of the technical solution of the present invention: from the product perspective, the present invention makes more refined use of hardware resources in the actual operation process, reduces the investment in related maintenance human resources, supplements the competition process, and makes supplementary improvements in functional design and independent deployment, thereby improving the evaluation efficiency, being able to evaluate and score in seconds, improving the user experience, and enabling users to intuitively feel the improvement of their own abilities during the competition, thereby greatly improving user stickiness and product reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the data competition automatic evaluation method provided by an embodiment of the present invention.

[0040] Figure 2 This is a flow chart of configuration and startup of an evaluation container provided by an embodiment of the present invention.

[0041] Figure 3 It is an evaluation flow chart provided by an embodiment of the present invention.

[0042] Figure 4 It is a schematic diagram of information included in a complete submitted task provided by an embodiment of the present invention.

[0043] Figure 5It is a schematic diagram of evaluation configuration information provided by an embodiment of the present invention.

[0044] Figure 6 After receiving the request, the Master node provided by the embodiment of the present invention will dynamically create an evaluation image according to the resource usage of each node in the cluster, and at the same time mount some information descriptions to the schematic diagram of the image environment in the form of environment variables.

[0045] Figure 7 It is a schematic diagram of establishing a connection between a scheduling image and an execution image using zmq via a TCP protocol and sharing information in a subsequent task process provided by an embodiment of the present invention.

[0046] Figure 8 It is a schematic diagram of a scheduling program provided in an embodiment of the present invention writing back information to a website through an encryption interface after receiving the information. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands and describes the technical solution of the claims.

[0049] like Figure 1 As shown, the data competition automatic evaluation method provided by the embodiment of the present invention includes:

[0050] S101, receiving the evaluation task, editing and saving the evaluation configuration according to the requirements of the competition, and then starting the evaluation environment through the scheduling program;

[0051] S102, automatically establish a queue monitor and download the corresponding file; and use the scheduling image and the execution image of the evaluation container to perform evaluation processing respectively to obtain an evaluation score.

[0052] After editing and saving the evaluation configuration according to the requirements of the competition topic provided in the embodiment of the present invention, before starting the evaluation environment through the scheduling program, the following steps need to be performed:

[0053] Upload the evaluation script and result set files of the competition questions in sequence through the management backend, and save the corresponding storage path, evaluation environment configuration, number of evaluation environment copies, and evaluation deadline into the database.

[0054] The corresponding files provided by the embodiment of the present invention include: an evaluation script, a standard file and a submission file.

[0055] The embodiment of the present invention provides a method for performing evaluation processing using a scheduling image and an execution image of an evaluation container respectively, including:

[0056] The scheduling image is used to download and save the task files, and process the evaluation information; the relevant information is shared with the execution image under the same container through the TCP protocol. After receiving the task information, the execution image scores the evaluation task and returns the result to the scheduling image after processing; the scheduling image returns the information to the scheduler through the queue for storage.

[0057] The embodiment of the present invention provides a method for performing evaluation processing using a scheduling image and an execution image of an evaluation container respectively, including the following steps:

[0058] (1) Use the scheduling image to establish a connection with the queue program and register the queue monitor. At the same time, the scheduling image downloads the saved evaluation script and result set file to the shared directory while ensuring that the connection is normal;

[0059] (2) The execution image and the scheduling image share information through the TCP protocol.

[0060] The data competition automatic evaluation method provided by the embodiment of the present invention also includes:

[0061] Submit the results through the website. After successful submission, the scheduler is triggered. When the scheduler starts, it creates a queue connection and sorts the information and delivers it to the message queue after receiving the evaluation task.

[0062] After the scheduling image listens to the new task through the queue, it will download the corresponding file, save the file to the specified directory and organize the evaluation information; after the scheduling image successfully saves the evaluation file, it will share the evaluation information to the execution image through zmq;

[0063] After the execution image obtains the evaluation information, it parses the content and scores the evaluation file through the evaluation script; the execution image shares the result to the scheduling image through zmq; the scheduling image obtains the information and sends the result back to the scheduler;

[0064] After receiving the information, the scheduler will write it back to the website through an encrypted interface.

[0065] The data competition automatic evaluation system provided by the embodiment of the present invention includes:

[0066] The storage module is used to store relevant documents and data through the scheduling program;

[0067] Evaluation configuration module, used to edit the evaluation configuration according to the requirements of the competition topic;

[0068] The evaluation environment startup module is used to start the evaluation environment through the scheduler;

[0069] Task execution module, used to execute evaluation tasks;

[0070] The anomaly detection module is used to detect task anomalies and automatically recover when an anomaly occurs in the task.

[0071] Embodiment 1:

[0072] Through the scheduling program as a transfer, the competition question evaluation script, result set file and image file path are stored in the management background. After editing and saving the evaluation configuration according to the competition question requirements, the evaluation environment is started through the scheduling program. After the evaluation environment is started, the queue monitoring and corresponding file download will be automatically established. After completing the above steps, it will remain silent until a task is delivered.

[0073] When the task is delivered, there will be two images in the container that do different things, which can be summarized as a scheduling image and an execution image. The scheduling image will download and save the task file first, and then process the evaluation information. After completion, the relevant information will be shared with the execution image in the same container through the TCP protocol. After receiving the task information, the execution image will score the evaluation task, and return the result to the scheduling image after processing. The scheduling image then returns the information to the scheduler through the queue to complete the warehousing operation.

[0074] Figure 2 This is a diagram of the configuration process and startup of a single-question evaluation environment.

[0075] Upload the test script and result set files of the competition questions in sequence through the management backend, and save the corresponding storage path, test environment configuration, number of test environment copies, and test deadline in the database;

[0076] After ensuring that task a) is completed, the management backend can access the scheduler through the corresponding switch to start the evaluation environment;

[0077] After the evaluation environment is started, it contains two execution images, namely the nodejs scheduling image and the python execution image;

[0078] After the Nodejs scheduling image is started, it will establish a connection with the queue program and register the queue listener

[0079] The Nodejs scheduling image downloads the evaluation script and result set files saved in step a) to the shared directory while ensuring that the connection is normal.

[0080] Python execution image and Nodejs scheduling image share information through TCP protocol

[0081] Figure 3 This is a schematic diagram of the single task evaluation process.

[0082] The user submits the result through the website, and the scheduler is triggered after the submission is successful.

[0083] The scheduler creates a queue connection when it starts, and after receiving the evaluation task, it organizes the information and delivers it to the message queue.

[0084] Nodejs schedules the image to download the corresponding file after listening to the new task through the queue

[0085] Nodejs schedules the image to save the file to the specified directory and organize the evaluation information

[0086] After the Nodejs scheduling image successfully saves the evaluation file, the evaluation information is shared to the Python execution image through zmq

[0087] Python executes the image to obtain the evaluation information, parses the content, and scores the evaluation file through the evaluation script

[0088] The Python execution image shares the result to the Nodejs scheduling image through zmq

[0089] Nodejs schedules the image to obtain information and sends the result back to the scheduler

[0090] After receiving the information, the scheduler will write it back to the website through the encrypted interface

[0091] Figure 4 The minimum information required for a single evaluation process after the website saves the user's submitted results. id is the unique index of the corresponding record, cmpt_id is the topic to which the task belongs, schedule_id is the corresponding stage of the task under the topic, target is the list of the corresponding stage, such as A list or B list, file_path is the save path of the file submitted by the task, the file is hosted through s3 cloud storage, team_id is the team id of the task, user_id is the user id of the task, score is the evaluation score, the value is empty when not evaluated, and 0 when the evaluation result is abnormal, status is the evaluation status of the task, -2 is pending evaluation, -1 is inferred to be queued, 0 is normal output result, 100 is abnormal evaluation status, errcode is the corresponding evaluation abnormal status code, 0 is normal, greater than 0 corresponds to abnormal status, message is the output information during the evaluation process, err_message is the abnormal output information during the evaluation process

[0092] Figure 5It is the configuration information of the evaluation environment, where id is the corresponding information index, cmpt_id is the contest ID associated with the configuration, end_time is the deregistration time of the evaluation environment. After the evaluation system is started, the system will automatically trigger the container destruction operation when it exceeds this time point. CPU is the CPU resource required for the evaluation image, memory is the memory resource required for the evaluation image, the scheduling image CPU is the default value, the memory is consistent with the python execution image, and the set value prevails. node_image_id is the scheduling image id, python_image_id is the execution image id, when any image is updated, the updated address can be stored in the database, and the corresponding image can be selected during configuration, replicas is the number of evaluation container copies, when the concurrent task volume is too large, multiple instances can be started through this item to quickly output results.

[0093] 2. Application Examples: In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.

[0094] The automated evaluation method provided by the embodiment of the present invention is applied to the data evaluation system of the data competition platform. The present invention can better complete the automated evaluation and result log output, so that the competition ranking can display the results fairly and impartially. For example, the competition topic "Semantic Retrieval and Intelligent Question and Answer of News Text Data" cites the automated evaluation system. The competition topic is divided into three stages. When the patent was written, the preliminary round had entered its end and was about to enter the semi-final. The automated evaluation system was able to run stably and smoothly throughout the preliminary round. From submitting files to returning results, it only takes an average of 4-5 seconds for the contestants to feedback the results to the users.

[0095] The following is the detailed configuration process of the embodiment:

[0096] (1) Select the corresponding league in the competition management list (ignore the league settings), create a competition for it, and fill in the basic content of the competition, including the competition name, competition description, competition type, reward type, reward settings, technical direction, industry direction, start and end time, and competition logo.

[0097] (2) Configure information about various units such as the organizer and other units associated with the competition topic.

[0098] (3) Configure a description of the competition topic, including the competition topic introduction, data evaluation instructions, and frequently asked questions.

[0099] (4) Configure the competition schedule information for the competition. Each competition schedule has an independent list and is strongly associated with the evaluation system. The schedule can be configured in a custom manner or directly inherited from the league.

[0100] (5) Other parameters of the competition, including the setting of competition authority and the setting of competition team members.

[0101] (6) After the topic configuration is completed, you can configure the evaluation information for the topic through the topic menu. The configuration entry can be found in the shortcut menu of the topic editing list.

[0102] (7) Add an evaluation description for the competition question, including the evaluation end time, CPU limit, and memory limit. Both CPU and memory are configurable parameters. Select the execution image. If the scheduling image is not necessary, the default is a fixed image. Set the number of simultaneous online evaluation instances. It is recommended to increase the number when the number of competition questions submitted is high or the list is switched, which will greatly increase the evaluation efficiency.

[0103] (8) The configuration of evaluation information is completed; data is configured for the competition question evaluation, including data association with the competition schedule, association with the ranking list under the competition schedule, data type, data description, and the file can be directly uploaded through the management background or filled in directly through the cloud disk address. The scoring script and answer file need to be configured.

[0104] (9) Image management, including image name, image description, and image address. After adding an image version, you can select the image you want to use in the image description.

[0105] (10) After the evaluation information is configured, you can start the evaluation image through the "Start Container" function button in the container configuration management.

[0106] (11) After the management backend is started, it sends a start request to the evaluation transfer service, which includes evaluation configuration information and evaluation question information;

[0107] After receiving the request, the evaluation transfer API will organize and summarize the evaluation information. After the organization is completed, it will provide an interface through kubernetes (hereinafter referred to as k8s) to send the startup information to the k8s master node

[0108] After receiving the request, the Master node will dynamically create an evaluation image based on the resource usage of each node in the cluster, and mount some information descriptions to the image environment in the form of environment variables. Figure 6 shown.

[0109] (12) After the evaluation image is started, the evaluation file is downloaded and placed in the shared directory according to the master preset information; the scheduling image establishes a listening relationship with the rabbit queue service and binds the competition question-specific queue service; the scheduling image and the execution image use the zmq protocol to establish a connection through the TCP protocol to share information in the subsequent task process, such as Figure 7 shown.

[0110] (13) Users register for the competition topic through the competition platform. After registration, they undergo some training or other operations to obtain a submittable file. The file is submitted through the competition platform. After successful submission, the scheduler is triggered. When the scheduler starts, a queue connection is created. After receiving the evaluation task, the information is sorted and delivered to the message queue.

[0111] After the scheduling image listens to the new task through the queue, it will download the corresponding file, save the file to the specified directory and organize the evaluation information; after the scheduling image successfully saves the evaluation file, it will share the evaluation information to the execution image through zmq;

[0112] After the execution image obtains the evaluation information, it parses the content and scores the evaluation file through the evaluation script; the execution image shares the result to the scheduling image through zmq; the scheduling image obtains the information and sends the result back to the scheduler;

[0113] After receiving the information, the scheduler will write it back to the website through the encrypted interface. Figure 8 shown.

[0114] The final evaluation score will be displayed under the competition question ranking list.

[0115] 3. Evidence of the effects of the embodiments. The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with the data, charts, etc. of the test process.

[0116] With the introduction of the current system, for competition questions with lighter tasks, the average evaluation time for the competition questions still reached 4.1 seconds, even including all the time spent on scoring and time-consuming processes such as file downloading.

[0117] When the scenario requires, the number of pod instances can be controlled through the management background during the peak phase, and the advantages of queues can be used to process multiple requests at the same time, with faster responses. Complex operations can be divided into multiple processes and performed simultaneously, and the evaluation requirements of the competition can be met through multi-tasking concurrency, greatly improving the evaluation efficiency.

[0118] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0119] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A data competition automated evaluation method, characterized in that: The data competition automatic evaluation method comprises: Receive the evaluation task, edit and save the evaluation configuration according to the requirements of the competition, and then start the evaluation environment through the scheduler; Automatically establish a queue monitor and download the corresponding files; use the scheduling image and execution image of the evaluation container to perform evaluation processing and obtain the evaluation score.

2. The data competition automated evaluation method according to claim 1, characterized in that: After editing and saving the evaluation configuration according to the requirements of the competition, before starting the evaluation environment through the scheduler, you need to do the following: Upload the evaluation script and result set files of the competition questions in sequence through the management backend, and save the corresponding storage path, evaluation environment configuration, number of evaluation environment copies, and evaluation deadline into the database.

3. The data competition automated evaluation method according to claim 1, characterized in that: The corresponding files include: evaluation scripts, standard files and submission files.

4. The data competition automated evaluation method according to claim 1, characterized in that: The use of the scheduling image and the execution image of the evaluation container to respectively perform evaluation processing includes: The scheduling image is used to download and save the task files, and process the evaluation information; the relevant information is shared with the execution image under the same container through the TCP protocol. After receiving the task information, the execution image scores the evaluation task and returns the result to the scheduling image after processing; the scheduling image returns the information to the scheduler through the queue for storage.

5. The data competition automated evaluation method according to claim 1, characterized in that: The use of the scheduling image and the execution image of the evaluation container to perform evaluation processing respectively includes the following steps: (1) Use the scheduling image to establish a connection with the queue program and register the queue monitor. At the same time, the scheduling image downloads the saved evaluation script and result set file to the shared directory while ensuring that the connection is normal; (2) The execution image and the scheduling image share information through the TCP protocol.

6. The data competition automated evaluation method according to claim 1, characterized in that: The data competition automatic evaluation method also includes: Submit the results through the website. After successful submission, the scheduler is triggered. When the scheduler starts, it creates a queue connection and sorts the information and delivers it to the message queue after receiving the evaluation task. After the scheduling image listens to the new task through the queue, it will download the corresponding file, save the file to the specified directory and organize the evaluation information; after the scheduling image successfully saves the evaluation file, it will share the evaluation information to the execution image through zmq; After the execution image obtains the evaluation information, it parses the content and scores the evaluation file through the evaluation script; the execution image shares the result to the scheduling image through zmq; the scheduling image obtains the information and sends the result back to the scheduler; After receiving the information, the scheduler will write it back to the website through an encrypted interface.

7. A data contest automated evaluation system implementing the data contest automated evaluation method according to any one of claims 1 to 6, characterized in that: The data competition automatic evaluation system includes: The storage module is used to store relevant documents and data through the scheduling program; Evaluation configuration module, used to edit the evaluation configuration according to the requirements of the competition topic; The evaluation environment startup module is used to start the evaluation environment through the scheduler; Task execution module, used to execute evaluation tasks; The anomaly detection module is used to detect task anomalies and automatically recover when an anomaly occurs in the task.

8. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps of the data competition automatic evaluation method according to any one of claims 1 to 6: Step 1: Receive the evaluation task, edit and save the evaluation configuration according to the requirements of the competition, and then start the evaluation environment through the scheduling program; Step 2: automatically establish a queue monitor and download the corresponding file; and use the scheduling image and execution image of the evaluation container to perform evaluation processing respectively to obtain the evaluation score.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps of the data competition automatic evaluation method according to any one of claims 1 to 6: Step 1: Receive the evaluation task, edit and save the evaluation configuration according to the requirements of the competition, and then start the evaluation environment through the scheduling program; Step 2: automatically establish a queue monitor and download the corresponding file; and use the scheduling image and execution image of the evaluation container to perform evaluation processing respectively to obtain the evaluation score.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the data competition automatic evaluation method as described in any one of claims 1-6.

Images