Competition management system

Abstract

The embodiment of the invention relates to a competition management system. The system comprises a sponsor submitting module, a competition storage module, a participant submitting module, a homework storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module, the sponsor submitting module is connected with the competition storage module; the competitor submitting module is connected with the competition storage module, the homework storage module and the competition management module; the match management module is connected with the homework storage module, the cross-platform computing cluster and the match storage module; and the match output module is connected with the match storage module. According to the method, the evaluation real-time performance and the evaluation efficiency can be improved, the consistency of the evaluation environment can be ensured, and the competition transparency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a competition management system. Background Art

[0002] Except for a few cup competitions or elite competitions that need to be conducted in a centralized competition format, most of the science and technology competitions related to software / algorithm development are managed in the form of open competitions, such as regular competitions held regularly by many universities, science and technology committees and other organizations. The traditional management method of this type of regular competition is: first publish the competition requirements through the Internet or journals, then receive the software / algorithm source code and corresponding documentation submitted by the contestants through email or network data upload, and then organize manual configuration of the operation environment and operation evaluation of the competition assignments submitted by each contestant, and finally give a list of winners based on the test results. Affected by human and equipment factors, the shortcomings of this traditional competition management method are obvious: the evaluation efficiency is very low, the consistency of the evaluation environment is difficult to ensure, and the transparency of the test process is very poor. The existence of these problems greatly reduces the enthusiasm of the contestants. Summary of the invention

[0003] The purpose of the present invention is to provide a competition management system to address the defects of the prior art. The system includes: an organizer submission module, a competition storage module, a contestant submission module, a job storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module. Among them, the organizer submission module customizes the competition rules through template configuration; contestants can submit their entries multiple times during the open period of the competition through the contestant submission module. The contestant submission module can also notify contestants in real time when the competition ranking changes, and can also provide backup data feedback to contestants every day; the event management module can conduct a real-time evaluation on each contestant after submitting his / her entry through linkage with the contestant submission module, and simultaneously refresh the real-time ranking list of all members; the event management module can also immediately conduct a synchronous evaluation of the final entry of all contestants after the time enters the closed period of the competition, and automatically give the final ultimate ranking list based on the evaluation results; the event management module can also use the platform host to build the same evaluation environment to implement the evaluation when evaluating any entry based on the mirror container technology; the event management module can also make cross-platform host calls when calling the platform host; the event output module is used to dynamically refresh the top rankings of the entire competition and provide a public web page for public display. The present invention can not only improve the real-time performance and efficiency of the evaluation, but also ensure the consistency of the evaluation environment and improve the transparency of the competition. Organizing competitions through the present invention can reduce the operational difficulty of participating in the competition on the one hand, and can also effectively improve the enthusiasm and initiative of the participants on the other hand.

[0004] To achieve the above-mentioned purpose, an embodiment of the present invention provides a competition management system, the system comprising: an organizer submission module, a competition storage module, a contestant submission module, a job storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module;

[0005] The organizer submission module is connected to the competition storage module; the contestant submission module is respectively connected to the competition storage module, the job storage module and the event management module; the event management module is respectively connected to the job storage module, the cross-platform computing cluster and the event storage module; the event output module is connected to the event storage module;

[0006] The organizer submission module is used to provide a competition configuration template to the competition organizer; and set corresponding data storage items for each template configuration item of the competition configuration template on the competition storage module; and store the setting content of each template configuration item in the corresponding data storage item; the template configuration item at least includes: organizer information configuration item, competition name configuration item, competition details configuration item, competition open period configuration item, competition closed period configuration item, job naming rule configuration item, job counting rule configuration item, job evaluation mode configuration item, job evaluation field set configuration item, job mirror file configuration item, mirror host parameter set configuration item, environment variable table configuration item, network whitelist configuration item, job test data set configuration item, job test script configuration item, job evaluation script configuration item, competition output rule configuration item; the data storage item at least includes: organizer information, competition name, competition details, competition open period, competition closed period, job naming rule, job counting rule, job evaluation mode, job evaluation field set, job mirror file, mirror host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rule;

[0007] The contestant submission module is used to allocate a corresponding contestant storage unit to each contestant on the homework storage module; and store the contestant identification and contestant graphic description corresponding to each contestant in the corresponding contestant storage unit; and manage the submission of the competition homework according to the competition opening period, the homework naming rules, the homework counting rules and the contestant storage unit corresponding to each contestant; and within the corresponding time period of the competition opening period, each successful completion of the submission of the competition homework will send an assessment scheduling request carrying the contestant identification corresponding to the homework to the competition management module; the contestant storage unit is used to store the contestant identification, the contestant graphic description, the latest submission time, the total number of submissions, the latest competition homework, the real-time evaluation queue, the ultimate evaluation record, the real-time ranking, and the ultimate ranking; the latest submission time, the latest competition homework, the real-time evaluation queue, and the ultimate evaluation record are all empty at the time of initialization; the initial value of the total number of submissions is 0;

[0008] The competition management module is used to store the corresponding organizer data items composed of the organizer information, the competition name and the competition details into the competition storage module; and set a real-time ranking table and a final ranking table on the competition storage module; the real-time and final ranking tables are composed of K ranking records, and the maximum total number of records on the list K is a positive integer; the list fields of the real-time and final ranking tables are composed of a contestant field, a total score field, multiple sub-item scoring fields, a submission time field and a submission number field; the contestant field corresponds to the contestant identifier; the sub-item scoring field corresponds to the homework evaluation field of the homework evaluation field set one by one; when the current time does not enter the corresponding time period of the competition closed period, all ranking records in the final ranking table are empty;

[0009] The competition management module is also used to extract the corresponding contestant identifier from the current evaluation scheduling request as the current contestant identifier each time a evaluation scheduling request is received; and perform real-time job evaluation and refresh the real-time ranking table based on the current contestant identifier, the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and the corresponding contestant storage unit;

[0010] The competition management module is also used to perform a final evaluation and set the final ranking table based on the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and all the contestant storage units after the current time enters the corresponding time period of the competition closed period;

[0011] The cross-platform computing cluster is composed of multiple first computing platforms; each of the first computing platforms is a computing center or a cloud computing platform; each of the first computing platforms includes multiple first platform hosts; each of the first platform hosts is a physical host or a virtual machine; each of the first platform hosts corresponds to a host configuration and a host operating condition; the host configuration includes at least the total number of CPU cores, memory, GPU configuration, GPU memory, and storage capacity; the GPU configuration includes available and not available; when the GPU configuration is not available, the GPU memory is empty; the host operating condition includes occupied and idle;

[0012] The event output module is used to configure the event signboard according to the event storage module.

[0013] Preferably, the organizer information is graphic and text information of the competition organizer;

[0014] The competition name mentioned is the name of this competition;

[0015] The competition details are graphic and text information of this competition;

[0016] The competition open period and the competition closed period are each a time period. During the competition open period, contestants can submit competition assignments multiple times, and during the competition closed period, contestants cannot submit competition assignments; the competition assignment is a Notebook file;

[0017] The job naming rules are used to uniformly set the file names of participating jobs;

[0018] The job counting rule is used to set one or more counting condition items and the condition judgment parameters corresponding to each of the counting condition items; and to set corresponding condition judgment symbols between each of the counting condition items and the corresponding condition judgment parameters, and the condition judgment symbols at least include equal to, not equal to, greater than or equal to, less than or equal to, include, and do not include; and each of the counting condition items and the corresponding condition judgment symbols and the condition judgment parameters constitute a corresponding counting condition judgment item; and to set a corresponding logical output state for each of the counting condition judgment items as the corresponding condition satisfaction state, and specify that the corresponding condition satisfaction state when each of the counting condition judgment items is established is logically true, otherwise it is logically false; and to set a corresponding logical operation formula for performing logical operations according to all the condition satisfaction states and outputting a corresponding logical operation result; the logical operators of the logical operation formula at least include or, and, not, exclusive or, and exclusive or; the logical operation result is logically true or logically false;

[0019] The operation evaluation mode includes data test + evaluation mode and single evaluation mode;

[0020] When the job evaluation mode is the data test + evaluation mode, it means that the participating job needs to be tested based on the job test data set and the participating job needs to be evaluated based on the test results; when the job evaluation mode is the data test + evaluation mode, the participating job should have a test code for referencing the data set with a specified name in the job evaluation directory in the job evaluation environment for testing, and saving the test results as the corresponding test output files in the job evaluation directory, and overwriting the previous test output files each time the test output files are saved;

[0021] When the job evaluation mode is the evaluation mode, it means that only the running result of the participating job needs to be evaluated; when the job evaluation mode is the evaluation mode, when the participating job is executed, the execution result will be saved as the corresponding test output file in the job evaluation directory, and each time the test output file is saved, the previous test output file will be overwritten;

[0022] The job evaluation field set is composed of one or more job evaluation fields; each of the job evaluation fields corresponds to a preset evaluation rule or evaluation algorithm;

[0023] The job image file is an installation package image file for building a competition job evaluation environment; the job image file encapsulates multiple sub-files and a main installation program; the multiple sub-files at least include an operating system component installation program, a Notebook server installation program, a Notebook programming language interpreter and a programming kernel installation program, a Notebook dependency library and extension library installation program, a job evaluation directory creation script, an environment variable table configuration program, and a job evaluation program; the main installation program is used to run each sub-file one by one in a specified execution or copy order; the operating system component installation program is executed by the main installation program and is used to install the operating system or operating system component set required for the competition job evaluation environment; the N The Notebook server installation program is executed by the main installation program and is used to install the Notebook server and all its corresponding service components; the Notebook programming language interpreter and programming kernel installation program are executed by the main installation program and are used to install programming language interpreters containing at least Python, Julia, and R languages ​​and corresponding programming kernel components; the job evaluation directory creation script is executed by the main installation program and is used to create a job evaluation directory under the root directory of the participating job evaluation environment; the environment variable table configuration program is copied by the main installation program to the root directory of the participating job evaluation environment; the job evaluation program is copied by the main installation program to the job evaluation directory of the participating job evaluation environment;

[0024] The environment variable table configuration program is used to set environment variables according to the environment variable table stored in the root directory when the program is running; the job evaluation program is used to perform corresponding evaluation field punching according to the evaluation rules or evaluation algorithms corresponding to each job evaluation field of the job evaluation field set and the test output files stored in the job evaluation directory to obtain corresponding evaluation field scores, and each job evaluation field and the corresponding evaluation field score form a corresponding evaluation field key-value pair, and all the obtained evaluation field key-value pairs form a corresponding job evaluation record and store it in the job evaluation directory;

[0025] The image host parameter set is a host hardware configuration parameter set for adapting the job image file, including at least the total number of host CPU cores, host memory, host GPU configuration options, host GPU memory, and storage capacity; the host GPU configuration options include selection and non-selection; when the host GPU configuration option is not selected, the host GPU memory is empty;

[0026] The environment variable table includes a plurality of environment variable key-value pairs, each of which consists of an environment variable name and an environment variable value;

[0027] When the network whitelist is empty, it indicates that there is no restriction on the available network segments of the cross-platform computing cluster; when the network whitelist is not empty, it is composed of one or more available network segments, and when selecting a platform host from the cross-platform computing cluster, only the available network segments provided by the whitelist can be selected;

[0028] When the operation evaluation mode is the data test + evaluation mode, the operation test data set is not empty; when the operation evaluation mode is the single evaluation mode, the operation test data set is empty;

[0029] The job test script is a shell instruction script for running the competition job in the competition job evaluation environment;

[0030] The job evaluation script is a shell instruction script for running the job evaluation program in the competition job evaluation environment;

[0031] The competition output rules include real-time ranking output rules and ultimate ranking output rules; the real-time ranking output rules correspond to a preset real-time total score calculation algorithm and are only applicable during the competition open period; the ultimate ranking output rules correspond to a preset ultimate total score calculation algorithm and are only applicable during the competition closed period.

[0032] Preferably, the contestant submission module is specifically used for managing the submission of the competition assignments according to the competition opening period, the assignment naming rules, the assignment counting rules, and the contestant storage unit corresponding to each contestant:

[0033] Step 31, receiving a first submission package sent by a contestant;

[0034] The first submission package consists of an obfuscated string and a competition work file; the obfuscated string is a string obtained by obfuscating a participant identifier based on a preset obfuscation algorithm; and the competition work file is a Notebook file;

[0035] Step 32, deobfuscate the obfuscated string based on a preset deobfuscation algorithm to obtain a corresponding first identifier; and use the contestant storage unit on the job storage module whose contestant identifier matches the first identifier as the corresponding current contestant storage unit; and identify whether the current contestant storage unit is empty or not unique; if the current contestant storage unit is empty or not unique, set the corresponding first submission status to failure and go to step 36; if the current contestant storage unit is not empty and unique, go to step 33;

[0036] Wherein, the anti-obfuscation algorithm and the obfuscation algorithm are inverse operations of each other;

[0037] Step 33, identifying whether the current time meets the competition open period; if not, setting the corresponding first submission status to failure and going to step 36; if satisfied, going to step 34;

[0038] Step 34, identifying whether the file name of the participating homework file meets the homework naming rule; if not, setting the corresponding first submission status to failure and going to step 36; if satisfied, going to step 35;

[0039] Step 35, identifying whether the logical operation result of the job counting rule is logically true; if not, setting the corresponding first submission status to failure; if so, setting the latest submission time of the current contestant storage unit to the current time, setting the latest entry job to the entry job file, and adding 1 to the total number of submissions of the current contestant storage unit, and setting the corresponding first submission status to success;

[0040] Step 36, feeding back the first submission status obtained to the current contestant.

[0041] Preferably, the competition management module is specifically used for performing real-time job evaluation and refreshing the real-time ranking table based on the current contestant identifier, the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and the corresponding contestant storage unit:

[0042] Step 41, taking the contestant storage unit whose contestant identification matches the current contestant identification as the corresponding current contestant storage unit; and taking the latest contestant assignment of the current contestant storage unit as the corresponding current contestant assignment;

[0043] Step 42, identifying the network whitelist; if the network whitelist is empty, selecting one of the first computing platforms in the cross-platform computing cluster whose host working condition is idle and whose host configuration satisfies the mirror host parameter set as the corresponding current host; if the network whitelist is not empty, recording each of the first computing platforms in the cross-platform computing cluster that satisfies the network whitelist as a corresponding matching platform, and selecting one of the first platform hosts whose host working condition is idle and whose host configuration satisfies the mirror host parameter set from all the matching platforms as the corresponding current host; and setting the host working condition of the current host to occupied;

[0044] Step 43, creating a corresponding first container based on the job image file on the current host; using the first container as the corresponding current competition job evaluation environment; copying the environment variable table to the root directory of the current competition job evaluation environment, and running the environment variable table configuration program in the root directory;

[0045] Step 44, if the job evaluation mode is the data test + evaluation mode, then extract a data subset from the head of the job test data set according to the preset first data extraction ratio as the corresponding current test data set; and name the current test data set according to the specified name to obtain the corresponding specified name data set; and copy the specified name data set, the job test script, the job evaluation script and the current participating job to the job evaluation directory of the current participating job evaluation environment in a manner of overwriting the original files; and run the job test script; and wait according to the preset first waiting time; and at the end of the first waiting time, identify whether the test output file in the job evaluation directory has been updated within the first waiting time; if so, run the job evaluation script, and extract the latest job evaluation record generated by the current script running in the job evaluation directory as the corresponding current job evaluation record; if not, generate a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 as the corresponding current job evaluation record; and add the current job evaluation record to the real-time evaluation queue of the current contestant storage unit;

[0046] The first data extraction ratio is 40% by default; the ratio of the data volume of the current test data set to the data volume of the job test data set satisfies the first data extraction ratio;

[0047] Step 45, if the job evaluation mode is a single evaluation mode, the job test script, the job evaluation script and the current participating job are copied to the job evaluation directory of the current participating job evaluation environment in a manner of overwriting the original files; and the job test script is run; and the second waiting time is preset to wait; and at the end of the second waiting time, it is identified whether the test output file in the job evaluation directory has been updated within the second waiting time; if so, the job evaluation script is run, and the latest job evaluation record generated by the current script running in the job evaluation directory is extracted as the corresponding current job evaluation record; if not, a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 is generated as the corresponding current job evaluation record; and the current job evaluation record is added to the real-time evaluation queue of the current contestant storage unit;

[0048] Step 46, deleting the first container; and releasing the occupied resources of the current image container creation operation on the current host; and setting the host working state of the current host to idle;

[0049] Step 47, extract the latest job evaluation record and the corresponding latest submission time and the total number of submissions in the real-time evaluation queue of each contestant storage unit on the job storage module to form a corresponding first evaluation data sequence; and calculate the total score according to each first evaluation data sequence using the real-time total score calculation algorithm corresponding to the real-time ranking output rule of the competition output rule to obtain the corresponding first total score; and sort the first total scores in descending order to form a corresponding first total score sequence; and record the K contestants corresponding to the first total scores in the first total score sequence that are in the first K positions as the first listed contestants; and use the sorting index corresponding to each of the first listed contestants in the first total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the real-time ranking table as the current ranking record, and based on the contestant identification, the first total score, the latest job evaluation record, the latest submission time and the total number of submissions of the current first listed contestant, the field values ​​of the corresponding contestant field, the total score field, the multiple sub-item scoring fields, the submission time field and the submission number field in the current ranking record are refreshed;

[0050] Step 48: taking the sorting index of the first total score corresponding to the current competition assignment in the first total score sequence as the corresponding current ranking; and resetting the real-time ranking in the current contestant storage unit based on the current ranking.

[0051] Preferably, the competition management module is specifically used for performing the final evaluation and setting the final ranking table based on the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and all the contestant storage units:

[0052] Step 51, counting the total number of the contestant storage units to obtain the corresponding total number of contestants N; and identifying the network whitelist; if the network whitelist is empty, selecting N first platform hosts whose host working conditions are idle and whose host configurations satisfy the mirror host parameter set from all the first computing platforms in the cross-platform computing cluster to form a corresponding final test host cluster; if the network whitelist is not empty, recording each of the first computing platforms that satisfy the network whitelist in the cross-platform computing cluster as a corresponding matching platform, and selecting N first platform hosts whose host working conditions are idle and whose host configurations satisfy the mirror host parameter set from all the matching platforms to form a corresponding final test host cluster; and setting the host working conditions of all the first platform hosts in the final test host cluster to be occupied;

[0053] Step 52, assign a corresponding first platform host to each of the contestant storage units as the corresponding contestant host; and create a corresponding second container based on the job image file on each of the contestant hosts; and use the latest contestant job in each of the contestant storage units as the corresponding final test contestant job; and use each of the second containers as the corresponding contestant job final test environment; and copy the environment variable table to the root directory of each contestant job final test environment; and run the environment variable table configuration program under the root directory of each contestant job final test environment;

[0054] Step 53, if the job evaluation mode is the data test + evaluation mode, a data subset is extracted from the tail of the job test data set as the corresponding final test data set according to the preset second data extraction ratio; and the final test data set is named according to the specified name to obtain the corresponding specified name data set; and the specified name data set, the job test script, the job evaluation script and each of the final test participating jobs are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each of the participating job final test environments; and in each of the participating job final test environments according to the specified name data set, the job test script, the job evaluation script and each of the final test participating jobs are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each of the participating job final test environments according to the specified name data set, and in each of the participating job final test environments according to the specified name data set, the job test script, the job evaluation script and each of the final test participating jobs are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files. Wait for a preset third waiting time, and at the end of the third waiting time, identify whether the test output file under the job evaluation directory of the current final test environment of the participating job has been updated within the third waiting time. If so, run the job evaluation script in the current job evaluation directory and extract the latest job evaluation record generated by the current script running as the corresponding final test evaluation record. If not, generate a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 as the corresponding final test evaluation record; and reset the ultimate evaluation record of each of the contestant storage units to the corresponding final test evaluation record;

[0055] The second data extraction ratio is 60% by default; the ratio of the data volume of the final test data set to the data volume of the operation test data set satisfies the second data extraction ratio;

[0056] Step 54, if the job evaluation mode is a single evaluation mode, the job test script, the job evaluation script and each of the final test participating jobs are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each of the participating job final test environments; and the fourth waiting time is waited for each of the participating job final test environments according to the preset fourth waiting time, and at the end of the fourth waiting time, it is identified whether the test output file in the job evaluation directory of the current participating job final test environment is updated within the fourth waiting time. If so, the job evaluation script is run in the current job evaluation directory and the latest job evaluation record generated by the current script running is extracted as the corresponding final test evaluation record. If not, a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 is generated as the corresponding final test evaluation record; and the ultimate evaluation record of each of the participant storage units is reset to the corresponding final test evaluation record;

[0057] Step 55, deleting all the second containers; releasing the resources occupied by the image container creation operation on all the contestant hosts; and setting the host working conditions of all the contestant hosts to idle;

[0058] Step 56, extract the ultimate evaluation record and the corresponding latest submission time and the total number of submissions of each of the contestant storage units on the job storage module to form a corresponding second evaluation data sequence; and calculate the total score according to each of the second evaluation data sequences using the ultimate total score calculation algorithm corresponding to the ultimate ranking output rule of the competition output rule to obtain the corresponding second total score; and sort the second total scores in descending order to form a corresponding second total score sequence; and record the K contestants corresponding to the second total scores that are in the first K positions in the second total score sequence as the second list; and use the sorting index corresponding to each of the second list in the second total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the ultimate ranking table as the current ranking record, and set the field values ​​of the corresponding contestant field, the total score field, the multiple sub-item scoring fields, the submission time field and the submission number field in the current ranking record based on the contestant identification, the second total score, the ultimate evaluation record, the latest submission time and the total number of submissions of the current second list;

[0059] Step 57, taking the sorting index of the second total score corresponding to each of the final test competition assignments in the second total score sequence as the corresponding final test ranking; and setting the ultimate ranking in each of the contestant storage units as the corresponding final test ranking.

[0060] Preferably, the event output module is specifically used to construct a web page as the corresponding event board page based on a preset event board page template when configuring the event board according to the event storage module; and publish the event board page on the public network; and regularly update the event information on the event board page based on the organizer data item, the real-time ranking table and the ultimate ranking table of the event storage module at a preset first refresh frequency; the page components of the event board page include an event name display area, an organizer information display area, an event details display area, a real-time ranking display area, and an ultimate ranking display area; the event name display area, the organizer information display area and the event details display area are used to display the corresponding competition name, the organizer information and the competition details description in the organizer data item of the event storage module; the real-time ranking display area is used to display the real-time ranking table of the event storage module; the ultimate ranking display area is used to display the ultimate ranking table of the event storage module.

[0061] Preferably, the contestant submission module is also used to extract the latest submission time, the total number of submissions, the latest job evaluation record in the real-time evaluation queue, the ultimate evaluation record, the real-time ranking and the ultimate ranking of the current contestant storage unit to form a corresponding update notification and send it to the contestant notification interface corresponding to the current contestant storage unit every time the real-time ranking or the ultimate ranking of each contestant storage unit is updated; the contestant notification interface includes at least an email interface and a SMS interface.

[0062] Preferably, the contestant submission module is also used to package the contestant identification, the contestant graphic description, the latest submission time, the total number of submissions, the latest entry assignment, the real-time evaluation queue, the ultimate evaluation record, the real-time ranking and the ultimate ranking of each contestant storage unit into a corresponding first backup data packet at a specified time every day and send it to the contestant data backup interface corresponding to the current contestant storage unit; the contestant data backup interface at least includes an email interface and a network storage interface.

[0063] An embodiment of the present invention provides a competition management system, which includes: an organizer submission module, a competition storage module, a contestant submission module, a job storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module. Among them, the organizer submission module customizes the competition rules through template configuration; contestants can submit their entries multiple times during the open period of the competition through the contestant submission module. The contestant submission module can also notify contestants in real time when the competition ranking changes, and can also provide backup data feedback to contestants every day; the event management module can conduct a real-time evaluation on each contestant after submitting his / her entry through linkage with the contestant submission module, and simultaneously refresh the real-time ranking list of all members; the event management module can also immediately conduct a synchronous evaluation of the final entry of all contestants after the time enters the closed period of the competition, and automatically give the final ultimate ranking list based on the evaluation results; the event management module can also use the platform host to build the same evaluation environment to implement the evaluation when evaluating any entry based on the mirror container technology; the event management module can also make cross-platform host calls when calling the platform host; the event output module is used to dynamically refresh the top rankings of the entire competition and provide a public web page for public display. The embodiment of the present invention not only improves the real-time performance and efficiency of the evaluation, but also ensures the consistency of the evaluation environment and improves the transparency of the competition. Organizing competitions through the embodiment of the present invention reduces the operational difficulty of the contestants on the one hand, and improves the enthusiasm and initiative of the contestants on the other hand. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A module structure diagram of a competition management system provided by an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of a real-time ranking table and a final ranking table provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] like Figure 1As shown in the module structure diagram of a competition management system provided by an embodiment of the present invention, a competition management system provided by an embodiment of the present invention mainly includes: an organizer submission module 1, a competition storage module 2, a contestant submission module 3, a job storage module 4, a competition management module 5, a cross-platform computing cluster 6, a competition storage module 7 and a competition output module 8. The connection relationship between the components of the system is: the organizer submission module 1 is connected to the competition storage module 2; the contestant submission module 3 is respectively connected to the competition storage module 2, the job storage module 4 and the competition management module 5; the competition management module 5 is respectively connected to the job storage module 4, the cross-platform computing cluster 6 and the competition storage module 7; the competition output module 8 is connected to the competition storage module 7.

[0068] 1. Organizer submission module 1, competition storage module 2:

[0069] The organizer submission module 1 of the embodiment of the present invention is used to provide a competition configuration template to the competition organizer; and set corresponding data storage items for each template configuration item of the competition configuration template on the competition storage module 2; and store the setting content of each template configuration item in the corresponding data storage item.

[0070] Here, the template configuration items of the competition configuration template of the embodiment of the present invention include at least: organizer information configuration item, competition name configuration item, competition details configuration item, competition open period configuration item, competition closed period configuration item, job naming rule configuration item, job counting rule configuration item, job evaluation mode configuration item, job evaluation field set configuration item, job image file configuration item, image host parameter set configuration item, environment variable table configuration item, network whitelist configuration item, job test data set configuration item, job test script configuration item, job evaluation script configuration item, and competition output rule configuration item.

[0071] Correspondingly, the data storage items on the competition storage module 2 include at least: organizer information, competition name, competition details, competition open period, competition closed period, job naming rules, job counting rules, job evaluation mode, job evaluation field set, job image file, image host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rules; among which:

[0072] 1) The organizer information is the graphic and text information of the competition organizer;

[0073] 2) The name of the competition is the name of this competition;

[0074] 3) Competition details include graphic and text information of the competition;

[0075] 4) The competition open period and the competition closed period are each a time period. During the competition open period, participants can submit their competition assignments multiple times, but during the competition closed period, participants cannot submit their competition assignments. The competition assignment is a Notebook file.

[0076] 5) Assignment Naming Rules: Unify the assignment file names for participating assignments;

[0077] 6) The operation counting rule is used to set one or more counting condition items and the condition judgment parameters corresponding to each counting condition item; and to set corresponding condition judgment symbols between each counting condition item and its corresponding condition judgment parameter, and the condition judgment symbols at least include equal to, not equal to, greater than or equal to, less than or equal to, include, and not include; and each counting condition item and its corresponding condition judgment symbol and condition judgment parameter constitute a corresponding counting condition judgment item; and to set a corresponding logical output state for each counting condition judgment item as the corresponding condition satisfaction state, and specify that the corresponding condition satisfaction state when each counting condition judgment item is established is logically true, otherwise it is logically false; and to set a corresponding logical operation formula for performing logical operations according to all condition satisfaction states and outputting a corresponding logical operation result; the logical operators of the logical operation formula at least include or, and, not, exclusive or, and exclusive or; the logical operation result is logically true or logically false;

[0078] 7) The homework assessment modes include data test + assessment mode and single assessment mode;

[0079] When the assignment evaluation mode is the data test + evaluation mode, it means that the assignment needs to be tested based on the assignment test data set and evaluated based on the test results. It should be noted that when the assignment evaluation mode is the data test + evaluation mode, the assignment should have a test code that references the data set with the specified name in the assignment evaluation directory in the assignment evaluation environment for testing, and saves the test results as the corresponding test output files in the assignment evaluation directory, and overwrites the previous test output files each time the test output files are saved.

[0080] When the job evaluation mode is the evaluation mode, it means that only the running results of the participating jobs need to be evaluated. It should be noted that when the job evaluation mode is the evaluation mode, the execution results will be saved as the corresponding test output files in the job evaluation directory when the participating jobs are executed, and the previous test output files will be overwritten each time the test output files are saved.

[0081] 8) The job evaluation field set consists of one or more job evaluation fields; each job evaluation field corresponds to a preset evaluation rule or evaluation algorithm;

[0082] Here, the evaluation rules or evaluation algorithms corresponding to each job evaluation field can be customized based on application requirements and introduced into the system in the form of rule / algorithm functions, interfaces or modules;

[0083] 9) The job image file is an installation package image file used to build the evaluation environment for the competition job;

[0084] It should be noted that the job image file encapsulates multiple sub-files and a main installation program; wherein the multiple sub-files at least include an operating system component installation program, a notebook server installation program, a notebook programming language interpreter and a programming kernel installation program, a notebook dependency library and extension library installation program, a job evaluation directory creation script, an environment variable table configuration program, and a job evaluation program; the main installation program is used to run each sub-file one by one in the specified execution or copy order;

[0085] Furthermore, the operating system component installation program is executed by the main installation program and used to install the operating system or operating system component set required for the participating homework evaluation environment; the Notebook server installation program is executed by the main installation program and used to install the Notebook server and all its corresponding service components; the Notebook programming language interpreter and programming kernel installation program are executed by the main installation program and used to install programming language interpreters containing at least Python, Julia, and R languages ​​and corresponding programming kernel components; the homework evaluation directory creation script is executed by the main installation program and used to create a homework evaluation directory under the root directory of the participating homework evaluation environment; the environment variable table configuration program is copied by the main installation program to the root directory of the participating homework evaluation environment; the homework evaluation program is copied by the main installation program to the homework evaluation directory of the participating homework evaluation environment;

[0086] It should also be noted that the above-mentioned environment variable table configuration program is used to set environment variables according to the environment variable table stored in the root directory when the program is running; the above-mentioned job evaluation program is used to perform the corresponding evaluation field according to the evaluation rules or evaluation algorithms corresponding to each job evaluation field of the job evaluation field set and the test output files stored in the job evaluation directory to obtain the corresponding evaluation field scores, and each job evaluation field and the corresponding evaluation field score form a corresponding evaluation field key-value pair, and all the obtained evaluation field key-value pairs form a corresponding job evaluation record and store it in the job evaluation directory;

[0087] 10) The image host parameter set is a host hardware configuration parameter set for adapting the job image file, including at least the total number of host CPU cores, host memory, host GPU configuration options, host GPU memory, and storage capacity; wherein the host GPU configuration option includes selection and non-selection; when the host GPU configuration option is not selected, the host GPU memory is empty;

[0088] 11) The environment variable table includes multiple environment variable key-value pairs, each of which consists of an environment variable name and an environment variable value;

[0089] 12) When the network whitelist is empty, it means that there is no restriction on the available network segments of the cross-platform computing cluster 6; when the network whitelist is not empty, it is composed of one or more available network segments, and when selecting a platform host from the cross-platform computing cluster 6, only the available network segments provided by the whitelist can be selected;

[0090] 13) When the job evaluation mode is data test + evaluation mode, the job test data set is not empty; when the job evaluation mode is single evaluation mode, the job test data set is empty;

[0091] 14) The job test script is a shell command script for running the competition job in the competition job evaluation environment;

[0092] 15) The job evaluation script is a shell command script for running the job evaluation program in the competition job evaluation environment;

[0093] 16) Competition output rules include real-time ranking output rules and ultimate ranking output rules; real-time ranking output rules correspond to a preset real-time total score calculation algorithm and are only applicable during the competition open period; ultimate ranking output rules correspond to a preset ultimate total score calculation algorithm and are only applicable during the competition closed period. Here, the real-time / ultimate total score calculation algorithm can be customized based on application requirements and introduced into the system in the form of rule / algorithm functions, interfaces or modules.

[0094] (II) Participant submission module 3, assignment storage module 4:

[0095] The contestant submission module 3 of the embodiment of the present invention is used to assign a corresponding contestant storage unit to each contestant on the assignment storage module 4; and store the contestant identification and contestant graphic description corresponding to each contestant in the corresponding contestant storage unit; and manage the submission of competition assignments according to the competition opening period, assignment naming rules, assignment counting rules and the contestant storage unit corresponding to each contestant; and within the corresponding time period of the competition opening period, each successful completion of the competition assignment submission will send an assessment scheduling request carrying the contestant identification corresponding to the assignment to the event management module 5.

[0096] The contestant submission module 3 is also used to extract the latest submission time, total number of submissions, latest job evaluation record in the real-time evaluation queue, ultimate evaluation record, real-time ranking and ultimate ranking of the current contestant storage unit every time the real-time ranking or ultimate ranking of each contestant storage unit is updated, and form a corresponding update notification to send to the contestant notification interface corresponding to the current contestant storage unit; the contestant notification interface includes at least an email interface and a SMS interface.

[0097] The contestant submission module 3 is also used to package the contestant identification, contestant graphic description, latest submission time, total number of submissions, latest entry assignment, real-time evaluation queue, ultimate evaluation record, real-time ranking and ultimate ranking of each contestant storage unit into a corresponding first backup data packet at a specified time every day and send it to the contestant data backup interface corresponding to the current contestant storage unit; the contestant data backup interface includes at least an email interface and a network storage interface.

[0098] The contestant storage unit on the job storage module 4 is used to store the contestant identification, contestant graphic description, latest submission time, total number of submissions, latest entry homework, real-time evaluation queue, ultimate evaluation record, real-time ranking, and ultimate ranking; among them, the latest submission time, the latest entry homework, the real-time evaluation queue, and the ultimate evaluation record are all empty during initialization; the initial value of the total number of submissions is 0.

[0099] In a specific implementation of the embodiment of the present invention, the contestant submission module 3 is specifically used to manage the submission of the competition assignments according to the competition opening period, assignment naming rules, assignment counting rules, and the contestant storage units corresponding to each contestant:

[0100] Step A1, receiving a first submission package sent by a contestant;

[0101] The first submission package consists of an obfuscated string and a competition assignment file; the obfuscated string is a string obtained by obfuscating a participant's ID based on a preset obfuscation algorithm; and the competition assignment file is a Notebook file;

[0102] Here, the obfuscation algorithm used in the embodiment of the present invention and the anti-obfuscation algorithm mentioned below are conventional obfuscation / anti-obfuscation algorithms, and can also be customized based on application requirements and introduced into the system in the form of rules / algorithm functions, interfaces or modules;

[0103] Step A2, deobfuscate the obfuscated character string based on a preset deobfuscation algorithm to obtain a corresponding first identifier; and use the contestant storage unit whose contestant identifier on the job storage module 4 matches the first identifier as the corresponding current contestant storage unit; and identify whether the current contestant storage unit is empty or not unique; if the current contestant storage unit is empty or not unique, set the corresponding first submission status to failure and go to step A6; if the current contestant storage unit is not empty and unique, go to step A3;

[0104] Among them, the anti-obfuscation algorithm and the obfuscation algorithm are inverse operations of each other;

[0105] Step A3, identifying whether the current time meets the competition open period; if not, setting the corresponding first submission status to failure and going to step A6; if satisfied, going to step A4;

[0106] Step A4, identifying whether the file name of the participating homework file meets the homework naming rules; if not, setting the corresponding first submission status to failure and going to step A6; if satisfied, going to step A5;

[0107] Step A5, identifying whether the logical operation result of the job counting rule is logically true; if not, setting the corresponding first submission status to failure; if yes, setting the latest submission time of the current contestant storage unit to the current time, setting the latest contestant job to the contestant job file, and adding 1 to the total number of submissions of the current contestant storage unit, and setting the corresponding first submission status to success;

[0108] Step A6: Feedback the obtained first submission status to the current contestant.

[0109] (III) Event management module 5, event storage module 7:

[0110] The competition management module 5 of the embodiment of the present invention is used to store the corresponding organizer data items consisting of organizer information, competition name and competition details into the competition storage module 7; and set a real-time ranking table and a final ranking table on the competition storage module 7.

[0111] Here, the event storage module 7 of the embodiment of the present invention is used to store the organizer data items, the real-time ranking table and the ultimate ranking table. Figure 2As shown in the schematic diagram of the real-time ranking table and the ultimate ranking table provided by the embodiment of the present invention, the real-time and ultimate ranking tables are composed of K ranking records, and the maximum total number of records K is a positive integer. The list fields of the real-time and ultimate ranking tables are composed of a contestant field, a total score field, multiple sub-item scoring fields, a submission time field, and a submission number field; among them, the contestant field corresponds to the contestant identifier; the sub-item scoring field corresponds to the homework evaluation field of the homework evaluation field set; when the current time does not enter the corresponding time period of the closed period of the competition, all ranking records of the ultimate ranking table are empty.

[0112] The event management module 5 is also used to extract the corresponding contestant ID from the current evaluation scheduling request as the current contestant ID each time an evaluation scheduling request is received; and based on the current contestant ID, job evaluation mode, job image file, image host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rules, cross-platform computing cluster 6 and corresponding contestant storage unit, perform real-time job evaluation and refresh the real-time ranking table.

[0113] The competition management module 5 is also used to perform a final evaluation and set a final ranking table based on the job evaluation mode, job image file, image host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rules, cross-platform computing cluster 6 and all contestant storage units after the current time enters the corresponding period of the competition closed period.

[0114] In another specific implementation of the embodiment of the present invention, the competition management module 5 is specifically used to perform real-time job evaluation and refresh the real-time ranking table based on the current contestant identification, job evaluation mode, job image file, image host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rules, cross-platform computing cluster 6 and corresponding contestant storage unit:

[0115] Step B1, taking the contestant storage unit whose contestant identification matches the current contestant identification as the corresponding current contestant storage unit; and taking the latest contestant assignment of the current contestant storage unit as the corresponding current contestant assignment;

[0116] Step B2, identifying the network whitelist; if the network whitelist is empty, then select a first platform host 611 whose host working condition is idle and whose host configuration satisfies the mirror host parameter set from all the first computing platforms 61 of the cross-platform computing cluster 6 as the corresponding current host; if the network whitelist is not empty, then each first computing platform 61 that satisfies the network whitelist in the cross-platform computing cluster 6 is recorded as the corresponding matching platform, and select a first platform host 611 whose host working condition is idle and whose host configuration satisfies the mirror host parameter set from all matching platforms as the corresponding current host; and set the host working condition of the current host to occupied;

[0117] Step B3, create a mirror container based on the job mirror file on the current host to obtain a corresponding first container; and use the first container as the corresponding current competition job evaluation environment; and copy the environment variable table to the root directory of the current competition job evaluation environment, and run the environment variable table configuration program in the root directory;

[0118] Step B4, if the job evaluation mode is the data test + evaluation mode, then extract a data subset from the head of the job test data set according to the preset first data extraction ratio as the corresponding current test data set; and name the current test data set according to the specified name to obtain the corresponding specified name data set; and copy the specified name data set, the job test script, the job evaluation script and the current participating job to the job evaluation directory of the current participating job evaluation environment in a manner of overwriting the original files; and run the job test script; and wait according to the preset first waiting time; and at the end of the first waiting time, identify whether the test output file in the job evaluation directory has been updated within the first waiting time; if so, run the job evaluation script, and extract the latest job evaluation record generated by the current script running in the job evaluation directory as the corresponding current job evaluation record; if not, generate a job evaluation record in which the evaluation field scores of all evaluation field key-value pairs are 0 as the corresponding current job evaluation record; and add the current job evaluation record to the real-time evaluation queue of the current contestant storage unit;

[0119] The first data extraction ratio is 40% by default; the ratio of the data volume of the current test data set to the data volume of the job test data set satisfies the first data extraction ratio; the first waiting time is a preset time length parameter;

[0120] Step B5, if the job evaluation mode is the single evaluation mode, the job test script, the job evaluation script and the current participating job are copied to the job evaluation directory of the current participating job evaluation environment in a manner of overwriting the original files; and the job test script is run; and the second waiting time is preset to wait; and at the end of the second waiting time, it is identified whether the test output file in the job evaluation directory has been updated within the second waiting time; if so, the job evaluation script is run, and the latest job evaluation record generated by the current script running in the job evaluation directory is extracted as the corresponding current job evaluation record; if not, a job evaluation record in which the evaluation field scores of all evaluation field key-value pairs are 0 is generated as the corresponding current job evaluation record; and the current job evaluation record is added to the real-time evaluation queue of the current contestant storage unit;

[0121] Here, the second waiting time is a preset time length parameter;

[0122] Step B6, deleting the first container; and releasing the resources occupied by the image container creation operation on the current host; and setting the host status of the current host to idle;

[0123] Step B7, extract the latest job evaluation record and the corresponding latest submission time and total number of submissions in the real-time evaluation queue of each contestant storage unit on the job storage module 4 to form a corresponding first evaluation data sequence; and calculate the total score according to each first evaluation data sequence using the real-time total score calculation algorithm corresponding to the real-time ranking output rule of the competition output rule to obtain the corresponding first total score; and sort the first total scores in order from high to low to form a corresponding first total score sequence; and record the K contestants corresponding to the first total scores in the first total score sequence that are in the top K positions as the first listed contestants; and use the sorting index corresponding to each first listed contestant in the first total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the real-time ranking table as the current ranking record, and based on the contestant identification, first total score, latest job evaluation record, latest submission time and total number of submissions of the current first listed contestant, the field values ​​of the corresponding contestant field, total score field, multiple sub-item scoring fields, submission time field and submission number field in the current ranking record are refreshed;

[0124] Step B8: taking the sorting index of the first total score corresponding to the current competition assignment in the first total score sequence as the corresponding current ranking; and resetting the real-time ranking in the current contestant storage unit based on the current ranking.

[0125] In another specific implementation of the embodiment of the present invention, the competition management module 5 is specifically used to perform the final evaluation and set the final ranking table based on the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rules, the cross-platform computing cluster 6 and all the contestant storage units:

[0126] Step C1, count the total number of contestant storage units to obtain the corresponding total number of contestants N; and identify the network whitelist; if the network whitelist is empty, select N first platform hosts 611 whose host conditions are idle and whose host configurations meet the mirror host parameter set from all first computing platforms 61 in the cross-platform computing cluster 6 to form a corresponding final test host cluster; if the network whitelist is not empty, record each first computing platform 61 that meets the network whitelist in the cross-platform computing cluster 6 as a corresponding matching platform, and select N first platform hosts 611 whose host conditions are idle and whose host configurations meet the mirror host parameter set from all matching platforms to form a corresponding final test host cluster; and set the host conditions of all first platform hosts 611 in the final test host cluster to be occupied;

[0127] Step C2, assign a corresponding first platform host 611 to each contestant storage unit as the corresponding contestant host; create a corresponding second container based on the job image file on each contestant host; use the latest contestant job in each contestant storage unit as the corresponding final test contestant job; use each second container as the corresponding contestant job final test environment; copy the environment variable table to the root directory of each contestant job final test environment; and run the environment variable table configuration program under the root directory of each contestant job final test environment;

[0128] Step C3, if the job evaluation mode is the data test + evaluation mode, then extract a data subset from the tail of the job test data set according to the preset second data extraction ratio as the corresponding final test data set; and name the final test data set according to the specified name to obtain the corresponding specified name data set; and copy the specified name data set, job test script, job evaluation script and each final test participating job to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original file; and run the job test script in the job evaluation directory of each participating job final test environment; and wait according to the preset third waiting time in each participating job final test environment, and at the end of the third waiting time, identify whether the test output file in the job evaluation directory of the current participating job final test environment has been updated within the third waiting time. If so, run the job evaluation script in the current job evaluation directory and extract the latest job evaluation record generated by the current script running as the corresponding final test evaluation record. If not, generate a job evaluation record in which the evaluation field scores of all evaluation field key-value pairs are 0 as the corresponding final test evaluation record; and reset the ultimate evaluation record of each participant storage unit to the corresponding final test evaluation record;

[0129] The second data extraction ratio is 60% by default; the ratio of the data volume of the final test data set to the data volume of the operation test data set satisfies the second data extraction ratio; the third waiting time is a preset time length parameter;

[0130] Step C4, if the job evaluation mode is the single evaluation mode, the job test script, the job evaluation script and each final test participating job are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each participating job final test environment; and each participating job final test environment is waited according to the preset fourth waiting time, and at the end of the fourth waiting time, it is identified whether the test output file in the job evaluation directory of the current participating job final test environment is updated within the fourth waiting time. If so, the job evaluation script is run in the current job evaluation directory and the latest job evaluation record generated by the current script running is extracted as the corresponding final test evaluation record. If not, a job evaluation record in which the evaluation field scores of all evaluation field key-value pairs are 0 is generated as the corresponding final test evaluation record; and the ultimate evaluation record of each participant storage unit is reset to the corresponding final test evaluation record;

[0131] Here, the fourth waiting time is a preset time length parameter;

[0132] Step C5, deleting all second containers; releasing the resources occupied by the mirror container creation operation on all contestant hosts; and setting the host status of all contestant hosts to idle;

[0133] Step C6, extract the ultimate evaluation records and the corresponding latest submission time and total number of submissions of each contestant storage unit on the job storage module 4 to form a corresponding second evaluation data sequence; and calculate the total score according to each second evaluation data sequence according to the ultimate total score calculation algorithm corresponding to the ultimate ranking output rule of the competition output rule to obtain the corresponding second total score; and sort the second total scores in order from high to low to form a corresponding second total score sequence; and record the K contestants corresponding to the second total scores in the first K positions in the second total score sequence as the second list; and use the sorting index corresponding to each second list in the second total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the ultimate ranking table as the current ranking record, and set the field values ​​of the corresponding contestant field, total score field, multiple sub-item scoring fields, submission time field and submission number field in the current ranking record based on the contestant identification, second total score, ultimate evaluation record, latest submission time and total number of submissions of the current second list;

[0134] Step C7, using the sorting index of the second total score corresponding to each final test competition assignment in the second total score sequence as the corresponding final test ranking; and setting the ultimate ranking in each contestant storage unit as the corresponding final test ranking.

[0135] (IV) Cross-platform computing cluster 6:

[0136] The cross-platform computing cluster 6 of the embodiment of the present invention is composed of multiple first computing platforms 61. Each first computing platform 61 is a computing center or a cloud computing platform. Each first computing platform 61 includes multiple first platform hosts 611. Each first platform host 611 is a physical host or a virtual machine. Each first platform host 611 corresponds to a host configuration and a host operating condition; the host configuration here includes at least the total number of CPU cores, memory, GPU configuration, GPU memory, and storage capacity; wherein, the GPU configuration includes having and not having; when the GPU configuration is not having, the GPU memory is empty; the host operating condition includes occupied and idle.

[0137] (V) Event output module 8:

[0138] The event output module 8 of the embodiment of the present invention is used to configure the event signboard according to the event storage module 7 .

[0139] In another specific implementation of the embodiment of the present invention, the event output module 8 is specifically used to construct a web page as the corresponding event board page based on the preset event board page template when configuring the event board according to the event storage module 7; and publish the event board page on the public network; and regularly update the event information on the event board page based on the organizer data items, real-time ranking table and ultimate ranking table of the event storage module 7 according to the preset first refresh frequency;

[0140] Here, the page components of the event kanban page of the embodiment of the present invention include an event name display area, a sponsor information display area, an event details display area, a real-time ranking display area, and an ultimate ranking display area; the event name display area, the sponsor information display area, and the event details display area are used to display the corresponding competition name, sponsor information, and competition details in the sponsor data item of the event storage module 7; the real-time ranking display area is used to display the real-time ranking table of the event storage module 7; and the ultimate ranking display area is used to display the ultimate ranking table of the event storage module 7. The first refresh frequency is a preset time frequency parameter.

[0141] It should be noted that the time management logic of the system of the embodiment of the present invention mentioned above is the time management logic of regular events, that is, a competition period is set and divided into an open period and a closed period. On this basis, regular events can be periodically replicated to form a long-term competition that is updated periodically. The specific technical implementation is to automatically copy a new competition storage module 2 based on the data items of the current competition storage module 2 every cycle, and initialize a new job storage module 4 and a new competition storage module 7 to form a competition storage set of a new round of cycles with this new competition storage module 2, and then the contestant submission module 3, the competition management module 5, the cross-platform computing cluster 6 and the competition output module 8 perform a new round of competition management based on the competition storage set of the new round of cycles according to the technical implementation steps of the system of the embodiment of the present invention. One thing to note here is that in this long-term competition mode, it is necessary to dynamically adjust the two time period parameters of the competition open period and the competition closed period in the competition storage module 2. This dynamic adjustment mechanism can be achieved by periodic accumulation without changing the relative duration ratio of the competition open period and the competition closed period. In addition, in the long-term competition mode, an achievement ranking table can be added to the event storage module 7 to rank all historical contestants. Correspondingly, it is necessary to add processing steps related to the achievement ranking table to the management function of the competition storage module 2, add page components related to the achievement ranking table to the event dashboard page output by the event output module 8, and add display processing steps related to the achievement ranking table to the management function of the event output module 8.

[0142] It should also be noted that, on the basis of the conventional competition mode, the competition cycle can be extended indefinitely, the competition open period can be extended indefinitely, the competition closed period can be cancelled or postponed indefinitely, the ultimate ranking table can be cancelled, and the scoring and issuing function can be added to convert it into a learning competition mode. The specific technical implementation is to set the competition open period on the competition storage module 2 to be forever, cancel or postpone the competition closed period indefinitely, and add a learning certificate template to the competition storage module 2; then cancel all data items and processing steps related to the ultimate ranking table on the operation storage module 4, the competition management module 5, the competition storage module 7, and the competition output module 8; and then add a scoring and issuing function to the competition management module 5, that is, when each contestant's real-time total score exceeds the set total score threshold (for example, 70 points) for the first time, a learning certificate is automatically generated based on the learning certificate template and the contestant information of the current contestant and sent to the current contestant's email notification interface.

[0143] It should also be noted that, in order to facilitate understanding, the embodiments of the present invention describe the technical implementation schemes of the above regular seasons, long-term competitions, and learning competitions based on a single event as a unit for competition management. In practical applications, based on the embodiments of the present invention, multiple competitions of different modes can be managed at the same time. Specifically, multiple competitions only need to be managed in parallel according to the competition management mechanism under each mode.

[0144] It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in software form; they can also all be implemented in hardware form; some modules can also be implemented in software form, and some modules can be implemented in hardware form. For example, the event management module can be a separately established processing device, equipment or server, or it can be integrated in a certain module of the above system for implementation. In addition, it can also be stored in the memory of the above system in the form of program code, and called and executed by a certain processing module of the above system. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing module described here can be an integrated circuit with signal processing capabilities. In the implementation process, the processing steps of each module of the aforementioned system can be completed by the integrated logic circuit of the hardware in the processing module or the instructions in the form of software.

[0145] For example, the modules of the above system may be one or more integrated circuits configured to implement the aforementioned system, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module of the above system is implemented in the form of a processing module scheduling program code, the processing module may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0146] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0147] An embodiment of the present invention provides a competition management system, which includes: an organizer submission module, a competition storage module, a contestant submission module, a job storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module. Among them, the organizer submission module customizes the competition rules through template configuration; contestants can submit their entries multiple times during the open period of the competition through the contestant submission module. The contestant submission module can also notify contestants in real time when the competition ranking changes, and can also provide backup data feedback to contestants every day; the event management module can conduct a real-time evaluation on each contestant after submitting his / her entry through linkage with the contestant submission module, and simultaneously refresh the real-time ranking list of all members; the event management module can also immediately conduct a synchronous evaluation of the final entry of all contestants after the time enters the closed period of the competition, and automatically give the final ultimate ranking list based on the evaluation results; the event management module can also use the platform host to build the same evaluation environment to implement the evaluation when evaluating any entry based on the mirror container technology; the event management module can also make cross-platform host calls when calling the platform host; the event output module is used to dynamically refresh the top rankings of the entire competition and provide a public web page for public display. The embodiment of the present invention not only improves the real-time performance and efficiency of the evaluation, but also ensures the consistency of the evaluation environment and improves the transparency of the competition. Organizing competitions through the embodiment of the present invention reduces the operational difficulty of the contestants on the one hand, and improves the enthusiasm and initiative of the contestants on the other hand.

[0148] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0149] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0150] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A competition management system, characterized in that: The system includes: an organizer submission module, a competition storage module, a contestant submission module, a job storage module, a competition management module, a cross-platform computing cluster, a competition storage module and a competition output module; The organizer submission module is connected to the competition storage module; the contestant submission module is respectively connected to the competition storage module, the job storage module and the event management module; the event management module is respectively connected to the job storage module, the cross-platform computing cluster and the event storage module; the event output module is connected to the event storage module; The organizer submission module is used to provide a competition configuration template to the competition organizer; and set corresponding data storage items for each template configuration item of the competition configuration template on the competition storage module; and store the setting content of each template configuration item in the corresponding data storage item; the template configuration item at least includes: organizer information configuration item, competition name configuration item, competition details configuration item, competition open period configuration item, competition closed period configuration item, job naming rule configuration item, job counting rule configuration item, job evaluation mode configuration item, job evaluation field set configuration item, job mirror file configuration item, mirror host parameter set configuration item, environment variable table configuration item, network whitelist configuration item, job test data set configuration item, job test script configuration item, job evaluation script configuration item, competition output rule configuration item; the data storage item at least includes: organizer information, competition name, competition details, competition open period, competition closed period, job naming rule, job counting rule, job evaluation mode, job evaluation field set, job mirror file, mirror host parameter set, environment variable table, network whitelist, job test data set, job test script, job evaluation script, competition output rule; The contestant submission module is used to allocate a corresponding contestant storage unit to each contestant on the homework storage module; and store the contestant identification and contestant graphic description corresponding to each contestant in the corresponding contestant storage unit; and manage the submission of the competition homework according to the competition opening period, the homework naming rules, the homework counting rules and the contestant storage unit corresponding to each contestant; and within the corresponding time period of the competition opening period, each successful completion of the submission of the competition homework will send an assessment scheduling request carrying the contestant identification corresponding to the homework to the competition management module; the contestant storage unit is used to store the contestant identification, the contestant graphic description, the latest submission time, the total number of submissions, the latest competition homework, the real-time evaluation queue, the ultimate evaluation record, the real-time ranking, and the ultimate ranking; the latest submission time, the latest competition homework, the real-time evaluation queue, and the ultimate evaluation record are all empty at the time of initialization; the initial value of the total number of submissions is 0; The competition management module is used to store the corresponding organizer data items composed of the organizer information, the competition name and the competition details into the competition storage module; and set a real-time ranking table and a final ranking table on the competition storage module; the real-time and final ranking tables are composed of K ranking records, and the maximum total number of records on the list K is a positive integer; the list fields of the real-time and final ranking tables are composed of a contestant field, a total score field, multiple sub-item scoring fields, a submission time field and a submission number field; the contestant field corresponds to the contestant identifier; the sub-item scoring field corresponds to the homework evaluation field of the homework evaluation field set one by one; when the current time does not enter the corresponding time period of the competition closed period, all ranking records in the final ranking table are empty; The competition management module is also used to extract the corresponding contestant identifier from the current evaluation scheduling request as the current contestant identifier each time a evaluation scheduling request is received; and perform real-time job evaluation and refresh the real-time ranking table based on the current contestant identifier, the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and the corresponding contestant storage unit; The competition management module is also used to perform a final evaluation and set the final ranking table based on the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and all the contestant storage units after the current time enters the corresponding time period of the competition closed period; The cross-platform computing cluster is composed of multiple first computing platforms; each of the first computing platforms is a computing center or a cloud computing platform; each of the first computing platforms includes multiple first platform hosts; each of the first platform hosts is a physical host or a virtual machine; each of the first platform hosts corresponds to a host configuration and a host operating condition; the host configuration includes at least the total number of CPU cores, memory, GPU configuration, GPU memory, and storage capacity; the GPU configuration includes available and not available; when the GPU configuration is not available, the GPU memory is empty; the host operating condition includes occupied and idle; The event output module is used to configure the event signboard according to the event storage module.

2. The competition management system according to claim 1, characterized in that: The organizer information is graphic and text information of the competition organizer; The competition name mentioned is the name of this competition; The competition details are graphic and text information of this competition; The competition open period and the competition closed period are each a time period. During the competition open period, contestants can submit competition assignments multiple times, and during the competition closed period, contestants cannot submit competition assignments; the competition assignment is a Notebook file; The job naming rules are used to uniformly set the file names of participating jobs; The operation counting rule is used to set one or more counting condition items and condition judgment parameters corresponding to each of the counting condition items; A corresponding conditional judgement symbol is set between each of the counting condition items and the corresponding conditional judgement parameter, and the conditional judgement symbol at least includes equal to, not equal to, greater than or equal to, less than or equal to, including, and not including; and each of the counting condition items and the corresponding conditional judgement symbol and the conditional judgement parameter constitutes a corresponding counting conditional judgement item; A corresponding logic output state is set for each of the counting condition judgment items as the corresponding condition satisfaction state, and the corresponding condition satisfaction state when each of the counting condition judgment items is established is designated as logical true, otherwise it is designated as logical false; and a corresponding logic operation formula is set for performing a logic operation according to all the condition satisfaction states and outputting a corresponding logic operation result; the logic operators of the logic operation formula at least include OR, AND, NOT, XOR, XNOR; the logic operation result is logical true or logical false; The operation evaluation mode includes data test + evaluation mode and single evaluation mode; When the job evaluation mode is the data test + evaluation mode, it means that the participating job needs to be tested based on the job test data set and the participating job needs to be evaluated based on the test results; when the job evaluation mode is the data test + evaluation mode, the participating job should have a test code for referencing the data set with a specified name in the job evaluation directory in the job evaluation environment for testing, and saving the test results as the corresponding test output files in the job evaluation directory, and overwriting the previous test output files each time the test output files are saved; When the job evaluation mode is the evaluation mode, it means that only the running result of the participating job needs to be evaluated; when the job evaluation mode is the evaluation mode, when the participating job is executed, the execution result will be saved as the corresponding test output file in the job evaluation directory, and each time the test output file is saved, the previous test output file will be overwritten; The job evaluation field set is composed of one or more job evaluation fields; each of the job evaluation fields corresponds to a preset evaluation rule or evaluation algorithm; The job image file is an installation package image file for building a competition job evaluation environment; the job image file encapsulates multiple sub-files and a main installation program; the multiple sub-files at least include an operating system component installation program, a Notebook server installation program, a Notebook programming language interpreter and a programming kernel installation program, a Notebook dependency library and extension library installation program, a job evaluation directory creation script, an environment variable table configuration program, and a job evaluation program; the main installation program is used to run each sub-file one by one in a specified execution or copy order; the operating system component installation program is executed by the main installation program and is used to install the operating system or operating system component set required for the competition job evaluation environment; the Notebook server installation program is executed by the main installation program and is used to install the Notebook server and all its corresponding service components; the Notebook programming language interpreter and programming kernel installation program are executed by the main installation program and are used to install programming language interpreters containing at least Python, Julia, and R languages ​​and corresponding programming kernel components; the job evaluation directory creation script is executed by the main installation program and is used to create a job evaluation directory under the root directory of the competition job evaluation environment; The environment variable table configuration program is copied by the main installation program to the root directory of the participating assignment evaluation environment; the assignment evaluation program is copied by the main installation program to the assignment evaluation directory of the participating assignment evaluation environment; The environment variable table configuration program is used to set environment variables according to the environment variable table stored in the root directory when the program is running; the job evaluation program is used to perform corresponding evaluation field punching according to the evaluation rules or evaluation algorithms corresponding to each job evaluation field of the job evaluation field set and the test output files stored in the job evaluation directory to obtain corresponding evaluation field scores, and each job evaluation field and the corresponding evaluation field score form a corresponding evaluation field key-value pair, and all the obtained evaluation field key-value pairs form a corresponding job evaluation record and store it in the job evaluation directory; The image host parameter set is a host hardware configuration parameter set for adapting the job image file, including at least the total number of host CPU cores, host memory, host GPU configuration options, host GPU memory, and storage capacity; the host GPU configuration options include selection and non-selection; when the host GPU configuration option is not selected, the host GPU memory is empty; The environment variable table includes a plurality of environment variable key-value pairs, each of which consists of an environment variable name and an environment variable value; When the network whitelist is empty, it indicates that there is no restriction on the available network segments of the cross-platform computing cluster; when the network whitelist is not empty, it is composed of one or more available network segments, and when selecting a platform host from the cross-platform computing cluster, only the available network segments provided by the whitelist can be selected; When the operation evaluation mode is the data test + evaluation mode, the operation test data set is not empty; when the operation evaluation mode is the single evaluation mode, the operation test data set is empty; The job test script is a shell instruction script for running the competition job in the competition job evaluation environment; The job evaluation script is a shell instruction script for running the job evaluation program in the competition job evaluation environment; The competition output rules include real-time ranking output rules and ultimate ranking output rules; the real-time ranking output rules correspond to a preset real-time total score calculation algorithm and are only applicable during the competition open period; the ultimate ranking output rules correspond to a preset ultimate total score calculation algorithm and are only applicable during the competition closed period.

3. The competition management system according to claim 2, characterized in that: The contestant submission module is specifically used for managing the submission of contest assignments according to the contest opening period, the assignment naming rules, the assignment counting rules, and the contestant storage unit corresponding to each contestant: Step 31, receiving a first submission package sent by a contestant; The first submission package consists of an obfuscated string and a competition work file; the obfuscated string is a string obtained by obfuscating a participant identifier based on a preset obfuscation algorithm; and the competition work file is a Notebook file; Step 32, deobfuscate the obfuscated string based on a preset deobfuscation algorithm to obtain a corresponding first identifier; and use the contestant storage unit on the job storage module whose contestant identifier matches the first identifier as the corresponding current contestant storage unit; and identify whether the current contestant storage unit is empty or not unique; if the current contestant storage unit is empty or not unique, set the corresponding first submission status to failure and go to step 36; if the current contestant storage unit is not empty and unique, go to step 33; Wherein, the anti-obfuscation algorithm and the obfuscation algorithm are inverse operations of each other; Step 33, identifying whether the current time meets the competition open period; if not, setting the corresponding first submission status to failure and going to step 36; if satisfied, going to step 34; Step 34, identifying whether the file name of the participating homework file meets the homework naming rule; if not, setting the corresponding first submission status to failure and going to step 36; if satisfied, going to step 35; Step 35, identifying whether the logical operation result of the job counting rule is logically true; if not, setting the corresponding first submission status to failure; if so, setting the latest submission time of the current contestant storage unit to the current time, setting the latest entry job to the entry job file, and adding 1 to the total number of submissions of the current contestant storage unit, and setting the corresponding first submission status to success; Step 36, feeding back the first submission status obtained to the current contestant.

4. The competition management system according to claim 2, characterized in that: The competition management module is specifically used for performing real-time job evaluation and refreshing the real-time ranking table based on the current contestant identifier, the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and the corresponding contestant storage unit: Step 41, taking the contestant storage unit whose contestant identification matches the current contestant identification as the corresponding current contestant storage unit; and taking the latest competition assignment in the current contestant storage unit as the corresponding current competition assignment; Step 42, identifying the network whitelist; If the network whitelist is empty, then one of the first computing platforms in the cross-platform computing cluster whose host working condition is idle and whose host configuration satisfies the mirror host parameter set is selected as the corresponding current host; if the network whitelist is not empty, then each of the first computing platforms in the cross-platform computing cluster that satisfies the network whitelist is recorded as a corresponding matching platform, and one of the first platform hosts whose host working condition is idle and whose host configuration satisfies the mirror host parameter set is selected from all the matching platforms as the corresponding current host; and setting the host operating state of the current host to occupied; Step 43, creating an image container based on the job image file on the current host to obtain a corresponding first container; and using the first container as the corresponding current competition job evaluation environment; and copying the environment variable table to the root directory of the current competition job evaluation environment, and running the environment variable table configuration program in the root directory; Step 44: if the job evaluation mode is a data test + evaluation mode, extract a data subset from the head of the job test data set as the corresponding current test data set according to a preset first data extraction ratio; And naming the current test data set according to the specified name to obtain the corresponding data set with the specified name; and copying the designated name data set, the job test script, the job evaluation script and the current entry job to the job evaluation directory of the current entry job evaluation environment in a manner of overwriting the original files; and running the job test script; And wait according to the preset first waiting time; and at the end of the first waiting time, identify whether the test output file in the job evaluation directory has been updated within the first waiting time; if so, run the job evaluation script, and extract the latest job evaluation record generated by the current script running in the job evaluation directory as the corresponding current job evaluation record; if not, generate a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 as the corresponding current job evaluation record; and add the current job evaluation record to the real-time evaluation queue of the current contestant storage unit; The first data extraction ratio is 40% by default; the ratio of the data volume of the current test data set to the data volume of the job test data set satisfies the first data extraction ratio; Step 45, if the job evaluation mode is a single evaluation mode, the job test script, the job evaluation script and the current participating job are copied to the job evaluation directory of the current participating job evaluation environment in a manner of overwriting the original files; and the job test script is run; and the second waiting time is preset to wait; and at the end of the second waiting time, it is identified whether the test output file in the job evaluation directory has been updated within the second waiting time; if so, the job evaluation script is run, and the latest job evaluation record generated by the current script running in the job evaluation directory is extracted as the corresponding current job evaluation record; if not, a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 is generated as the corresponding current job evaluation record; and the current job evaluation record is added to the real-time evaluation queue of the current contestant storage unit; Step 46, deleting the first container; and releasing the resources occupied by the image container creation operation on the current host; and setting the host status of the current host to idle; Step 47, extract the latest job evaluation record and the corresponding latest submission time and the total number of submissions in the real-time evaluation queue of each contestant storage unit on the job storage module to form a corresponding first evaluation data sequence; and calculate the total score according to each first evaluation data sequence using the real-time total score calculation algorithm corresponding to the real-time ranking output rule of the competition output rule to obtain the corresponding first total score; and sort the first total scores in descending order to form a corresponding first total score sequence; and record the K contestants corresponding to the first total scores in the first total score sequence that are in the first K positions as the first listed contestants; and use the sorting index corresponding to each of the first listed contestants in the first total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the real-time ranking table as the current ranking record, and based on the contestant identification, the first total score, the latest job evaluation record, the latest submission time and the total number of submissions of the current first listed contestant, the field values ​​of the corresponding contestant field, the total score field, the multiple sub-item scoring fields, the submission time field and the submission number field in the current ranking record are refreshed; Step 48: taking the sorting index of the first total score corresponding to the current competition assignment in the first total score sequence as the corresponding current ranking; and resetting the real-time ranking in the current contestant storage unit based on the current ranking.

5. The competition management system according to claim 2, characterized in that: The competition management module is specifically used for performing the final evaluation and setting the final ranking table based on the job evaluation mode, the job image file, the image host parameter set, the environment variable table, the network whitelist, the job test data set, the job test script, the job evaluation script, the competition output rule, the cross-platform computing cluster and all the contestant storage units: Step 51, counting the total number of the contestant storage units to obtain the corresponding total number of contestants N; and identifying the network whitelist; If the network whitelist is empty, N first platform hosts whose host working conditions are idle and whose host configurations satisfy the mirror host parameter set are selected from all the first computing platforms in the cross-platform computing cluster to form a corresponding final test host cluster; if the network whitelist is not empty, each of the first computing platforms that satisfy the network whitelist in the cross-platform computing cluster is recorded as a corresponding matching platform, and N first platform hosts whose host working conditions are idle and whose host configurations satisfy the mirror host parameter set are selected from all the matching platforms to form a corresponding final test host cluster; and setting the host working status of all the first platform hosts in the final test host cluster to occupied; Step 52, allocating a corresponding first platform host to each of the contestant storage units as a corresponding contestant host; And creating an image container based on the job image file on each of the contestant hosts to obtain a corresponding second container; and taking the latest competition assignment in each of the competition storage units as the corresponding final test competition assignment; and using each of the second containers as the corresponding final test environment for the participating job; and copying the environment variable table to the root directory of each of the final test environments for the participating job; and running the environment variable table configuration program in the root directory of each of the final test environments for the participating job; Step 53: if the operation evaluation mode is the data test + evaluation mode, extract a data subset from the tail of the operation test data set as the corresponding final test data set according to the preset second data extraction ratio; and name the final test data set according to the specified name to obtain the corresponding specified name data set; The designated name data set, the job test script, the job evaluation script and each of the final test competition jobs are copied to the job evaluation directory of the corresponding competition job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each of the competition job final test environments; And wait according to the preset third waiting time in each of the participating assignment final test environments, and at the end of the third waiting time, identify whether the test output file in the assignment evaluation directory of the current participating assignment final test environment has been updated within the third waiting time, if so, run the assignment evaluation script in the current assignment evaluation directory and extract the latest assignment evaluation record generated by the current script running as the corresponding final test evaluation record, if not, generate a assignment evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 as the corresponding final test evaluation record; and reset the ultimate evaluation record of each of the contestant storage units to the corresponding final test evaluation record; The second data extraction ratio is 60% by default; the ratio of the data volume of the final test data set to the data volume of the operation test data set satisfies the second data extraction ratio; Step 54, if the job evaluation mode is a single evaluation mode, the job test script, the job evaluation script and each of the final test participating jobs are copied to the job evaluation directory of the corresponding participating job final test environment in a manner of overwriting the original files; and the job test script is run in the job evaluation directory of each of the participating job final test environments; and the fourth waiting time is waited for each of the participating job final test environments according to the preset fourth waiting time, and at the end of the fourth waiting time, it is identified whether the test output file in the job evaluation directory of the current participating job final test environment is updated within the fourth waiting time. If so, the job evaluation script is run in the current job evaluation directory and the latest job evaluation record generated by the current script running is extracted as the corresponding final test evaluation record. If not, a job evaluation record in which the evaluation field scores of all the evaluation field key-value pairs are 0 is generated as the corresponding final test evaluation record; and the ultimate evaluation record of each of the participant storage units is reset to the corresponding final test evaluation record; Step 55, deleting all the second containers; releasing the resources occupied by the image container creation operation on all the contestant hosts; and setting the host working conditions of all the contestant hosts to idle; Step 56, extract the ultimate evaluation record and the corresponding latest submission time and the total number of submissions of each of the contestant storage units on the job storage module to form a corresponding second evaluation data sequence; and calculate the total score according to each of the second evaluation data sequences using the ultimate total score calculation algorithm corresponding to the ultimate ranking output rule of the competition output rule to obtain the corresponding second total score; and sort the second total scores in descending order to form a corresponding second total score sequence; and record the K contestants corresponding to the second total scores that are in the first K positions in the second total score sequence as the second list; and use the sorting index corresponding to each of the second list in the second total score sequence as the corresponding current ranking, and use the ranking record corresponding to the record index and the current ranking in the ultimate ranking table as the current ranking record, and set the field values ​​of the corresponding contestant field, the total score field, the multiple sub-item scoring fields, the submission time field and the submission number field in the current ranking record based on the contestant identification, the second total score, the ultimate evaluation record, the latest submission time and the total number of submissions of the current second list; Step 57, taking the sorting index of the second total score corresponding to each of the final test competition assignments in the second total score sequence as the corresponding final test ranking; and setting the ultimate ranking in each of the contestant storage units as the corresponding final test ranking.

6. The competition management system according to claim 1, characterized in that: The event output module is specifically used to construct a web page as a corresponding event board page based on a preset event board page template when configuring the event board according to the event storage module; and publish the event board page on the public network; and regularly update the event information on the event board page based on the organizer data item, the real-time ranking table and the ultimate ranking table of the event storage module at a preset first refresh frequency; the page components of the event board page include an event name display area, an organizer information display area, an event details display area, a real-time ranking display area, and an ultimate ranking display area; the event name display area, the organizer information display area and the event details display area are used to display the corresponding competition name, the organizer information and the competition details description in the organizer data item of the event storage module; the real-time ranking display area is used to display the real-time ranking table of the event storage module; the ultimate ranking display area is used to display the ultimate ranking table of the event storage module.

7. The competition management system according to claim 1, characterized in that: The contestant submission module is also used to extract the latest submission time, the total number of submissions, the latest job evaluation record in the real-time evaluation queue, the ultimate evaluation record, the real-time ranking and the ultimate ranking of the current contestant storage unit each time the real-time ranking or the ultimate ranking of each contestant storage unit is updated to form a corresponding update notification and send it to the contestant notification interface corresponding to the current contestant storage unit; the contestant notification interface includes at least an email interface and a SMS interface.

8. The competition management system according to claim 1, characterized in that: The contestant submission module is also used to package the contestant identification, the contestant graphic description, the latest submission time, the total number of submissions, the latest entry assignment, the real-time evaluation queue, the ultimate evaluation record, the real-time ranking and the ultimate ranking of each contestant storage unit into a corresponding first backup data packet at a specified time every day and send it to the contestant data backup interface corresponding to the current contestant storage unit; the contestant data backup interface at least includes an email interface and a network storage interface.

Images