Information processing system, information processing method, sorting device, notification device, sorting program, and notification program

Abstract

To provide an information processing system, an information processing method, an assignment apparatus, a notification apparatus, an assignment program, and a notification program that reduce a time period required for feedback of an evaluation result regarding a program code.SOLUTION: A method includes calculating a first evaluation result regarding a first program code generated in the past by a user who has generated each program code for a problem corresponding to each program code, for each program code stored in one of a plurality of queues corresponding to each of a plurality of evaluation apparatuses performing evaluation of the program code when accepting a new program code, and a second evaluation result regarding a second program code generated in the past using a programming language corresponding to each program code for the problem corresponding to each program code, and specifying a first queue storing the new program code from the plurality of queues and storing the same based on the first evaluation result and the second evaluation result.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, a distribution device, a notification device, a distribution program, and a notification program.

Background Art

[0002] In recent years, as a tool for supporting programming education for users, an evaluation device capable of automatically evaluating program codes has been widely used.

[0003] Such an evaluation device, for example, when receiving a submission of a program code to be evaluated, evaluates the program code by compiling and executing the received program code. Then, the evaluation device, for example, feeds back the evaluation result of the received program code to the user (for example, the creator of the received program code).

[0004] Thereby, the user can, for example, improve their programming ability by referring to the feedback from the evaluation device (see Patent Documents 1 to 2 and Non-Patent Documents 1 to 2).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0006]

Non-Patent Document 1

Non-Patent Document 2

[0007] In this context, it is preferable for the evaluation device described above to promptly provide feedback on the evaluation results of the submitted program code once it has received the program code to be evaluated. Therefore, the administrator of the evaluation device may, for example, operate multiple evaluation devices in parallel and also operate a distribution device that distributes the program code to be evaluated to multiple evaluation devices. This makes it possible to provide prompt feedback on the evaluation results of the program code, even if a large amount of program code is submitted.

[0008] However, as mentioned above, even when multiple evaluation devices share the task of evaluating program code, if the method of distributing the program code among the multiple evaluation devices is inappropriate, some evaluation devices may end up with more program code than others, resulting in delays in providing feedback on the evaluation results to the user.

[0009] Therefore, the objective of the present invention is to reduce the time required for feedback of evaluation results regarding program code. but shortening This may cause The objective is to provide an information processing system, information processing method, sorting device, notification device, sorting program, and notification program that enable this. [Means for solving the problem]

[0010] To achieve the above objective, the present invention provides an information processing system comprising: a plurality of evaluation devices, each performing an evaluation of a program code created by a user; a plurality of queues corresponding to each of the plurality of evaluation devices; a distribution device that distributes the program code to one of the plurality of queues; and a notification device that notifies the source of the program code of information about the program code stored in one of the plurality of queues, wherein when the distribution device receives a new program code, for each program code stored in one of the plurality of queues, it examines the first program code previously created by the user who created each program code for the problem corresponding to each program code. The system calculates a first evaluation result from the multiple evaluation devices and a second evaluation result from the multiple evaluation devices for a second program code previously created using the programming language corresponding to each program code for a problem corresponding to each program code. Based on the first and second evaluation results for each program code stored in one of the multiple queues, the system identifies a first queue for storing the new program code from the multiple queues, stores the new program code in the identified first queue, and notifies the notification destination corresponding to the user who generated the new program code of the information regarding the new program code stored in the identified first queue. [Effects of the Invention]

[0011] According to the information processing system, information processing method, sorting device, notification device, sorting program, and notification program of the present invention, the time required for feedback of evaluation results for program code is reduced. but shortening This may cause It becomes possible to do so. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 shows an example of the configuration of the information processing system 10 in the first embodiment. [Figure 2] FIG. 2 is a diagram showing a configuration example of the distribution device 1 in the first embodiment. [Figure 3] FIG. 3 is a diagram showing a hardware configuration example of the distribution device 1 in the first embodiment. [Figure 4] FIG. 4 is a diagram showing a hardware configuration example of the notification device 2 in the first embodiment. [Figure 5] FIG. 5 is a diagram showing a hardware configuration example of the evaluation device 3 in the first embodiment. [Figure 6] FIG. 6 is a diagram for explaining an outline of the code distribution process in the first embodiment. [Figure 7] FIG. 7 is a diagram for explaining an outline of the information notification process in the first embodiment. [Figure 8] FIG. 8 is a flowchart for explaining details of the code distribution process in the first embodiment. [Figure 9] FIG. 9 is a flowchart for explaining details of the information notification process in the first embodiment. [Figure 10] FIG. 10 is a flowchart for explaining details of the information notification process in the first embodiment. [Figure 11] FIG. 11 is a flowchart for explaining details of the information notification process in the first embodiment. [Figure 12] FIG. 12 is a diagram for explaining details of the information notification process in the first embodiment.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, such embodiment examples do not limit the technical scope of the present invention.

[0014] [Configuration of Information Processing System 10] First, we will describe an example of the configuration of the information processing system 10 in the first embodiment. Figure 1 is a diagram showing an example of the configuration of the information processing system 10 in the first embodiment. Figure 2 is a diagram showing an example of the configuration of the distribution device 1 in the first embodiment.

[0015] As shown in Figure 1, the information processing system 10 includes, for example, a distribution device 1, a notification device 2, evaluation devices 3a, 3b, and 3c, and user terminals 5a, 5b, and 5c. Furthermore, as shown in Figure 2, the distribution device 1 includes a processing unit 1a that performs each processing task, and queues 4a, 4b, and 4c corresponding to evaluation devices 3a, 3b, and 3c, respectively. Hereinafter, evaluation devices 3a, 3b, and 3c will be collectively referred to as evaluation device 3, queues 4a, 4b, and 4c will be collectively referred to as queue 4, and user terminals 5a, 5b, and 5c will be collectively referred to as user terminal 5. The following description will assume that the information processing system 10 has three evaluation devices 3 and three user terminals 5; however, the information processing system 10 may have a different number of evaluation devices 3 and user terminals 5.

[0016] User terminal 5 is, for example, a PC (Personal Computer) or a mobile device such as a smartphone owned by each user. The user then transmits program code created for a pre-specified problem (hereinafter simply referred to as the problem) to the distribution device 1 via user terminal 5.

[0017] The distribution device 1 determines, for example, which evaluation device 3 will distribute the program code sent from the user terminal 5. Then, the distribution device 1 stores, for example, the program code sent from the user terminal 5 in the queue 4 corresponding to the determined evaluation device 3.

[0018] The evaluation device 3 receives program code stored in queue 4 corresponding to itself from the distribution device 1 in the order in which it was stored. Then, the evaluation device 3 performs an evaluation of the received program code. Specifically, the evaluation device 3 performs an evaluation of the received program code by compiling and executing the received program code. After that, the evaluation device 3 sends the evaluation results of the received program code to the user terminal 5 that sent the received program code. The evaluation results of the program code include information such as whether the program code is correct or incorrect and information about the amount of resources used during the execution of the program code (e.g., CPU usage, memory usage, etc.).

[0019] Notification device 2 notifies the user who created each program code of information including the remaining time (hereinafter also referred to as waiting time) required for the evaluation of each program code stored in queue 4 to be completed. Specifically, notification device 2 obtains information including the waiting time for evaluation of each program code stored in queue 4 from distribution device 1. Then, notification device 2 transmits each piece of information obtained from distribution device 1 to the user terminal 5 that sent each program code. In addition, notification device 2 transmits the evaluation results for each program code evaluated by evaluation device 3 to the user terminal 5 that sent each program code.

[0020] [Hardware configuration of distribution device 1] Next, we will describe an example of the hardware configuration of the sorting device 1 in the first embodiment. Figure 3 is a diagram showing an example of the hardware configuration of the sorting device 1 in the first embodiment.

[0021] The distribution device 1 is a computer device, such as a general-purpose PC (Personal Computer) or a server device. The distribution device 1 performs various processes, including, for example, the process of distributing program code sent from the user terminal 5 to the evaluation device 3 (hereinafter also referred to as the code distribution process).

[0022] The distribution device 1 has a hardware configuration similar to that of a general-purpose computer device, and for example, as shown in Figure 3, it includes a processor CPU 101, memory 102, a communication interface 103, and a storage medium 104. Each part is connected to the others via a bus 105.

[0023] The storage medium 104 has, for example, a program storage area (not shown) for storing a program (not shown) for performing code distribution processing.

[0024] Furthermore, the storage medium 104 has a storage unit 110 (hereinafter also referred to as the storage area 110) that stores information used when performing code distribution processing. The storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0025] The CPU 101 executes the program loaded from the storage medium 104 into memory 102 and performs code distribution processing.

[0026] The communication interface 103 communicates with, for example, the evaluation device 3. The communication interface 103 also communicates with the user terminal 5 via a network NW, such as the Internet.

[0027] [Hardware configuration of notification device 2] Next, we will describe an example of the hardware configuration of the notification device 2 in the first embodiment. Figure 4 is a diagram showing an example of the hardware configuration of the notification device 2 in the first embodiment.

[0028] Notification device 2 is a computer device, such as a general-purpose PC or server device. Notification device 2 performs various processes, including, for example, a process that notifies information indicating the progress of evaluation for each program code stored in queue 4 (hereinafter also referred to as information notification process).

[0029] The notification device 2 has a general-purpose computer hardware configuration, and for example, as shown in Figure 4, it includes a processor CPU 201, memory 202, a communication interface 203, and a storage medium 204. Each part is connected to the others via a bus 205.

[0030] The storage medium 204 has, for example, a program storage area (not shown) for storing a program (not shown) for performing information notification processing.

[0031] Furthermore, the storage medium 204 has, for example, a storage unit 210 (hereinafter also referred to as the storage area 210) that stores information used when performing information notification processing. The storage medium 204 may be, for example, an HDD or an SSD.

[0032] The CPU 201 executes the program loaded from the storage medium 204 into memory 202 to perform information notification processing.

[0033] The communication interface 203 communicates with, for example, the evaluation device 3. The communication interface 203 also communicates with the user terminal 5 via a network NW, such as the Internet.

[0034] [Hardware configuration of evaluation device 3] Next, we will describe an example of the hardware configuration of the evaluation device 3 in the first embodiment. Figure 5 is a diagram showing an example of the hardware configuration of the evaluation device 3 in the first embodiment.

[0035] The evaluation device 3 is a computer device, such as a general-purpose PC or server device. The evaluation device 3 has the hardware configuration of a general-purpose computer device, and for example, as shown in Figure 5, it has a processor CPU 301, memory 302, communication interface 303, and storage medium 304. Each part is connected to the others via a bus 305.

[0036] The storage medium 304 has, for example, a program storage area (not shown) for storing programs (not shown) for performing various processes.

[0037] Furthermore, the storage medium 304 has, for example, a storage unit 310 (hereinafter also referred to as the storage area 310) that stores information used when performing various processes. The storage medium 304 may be, for example, an HDD or an SSD.

[0038] The CPU 301 executes the program loaded from the storage medium 304 into memory 302 and performs various processes.

[0039] The communication interface 303 communicates with, for example, the distribution device 1 and the notification device 2.

[0040] [Outline of the First Embodiment] Figure 6 is a diagram illustrating the outline of the code distribution process in the first embodiment. Figure 7 is a diagram illustrating the outline of the information notification process in the first embodiment.

[0041] First, we will describe the outline of the code distribution process in the first embodiment.

[0042] As shown in Figure 6, the code receiving unit 111 of the distribution device 1 receives program codes transmitted from, for example, the user terminal 5.

[0043] The time calculation unit 112 (hereinafter also referred to as the result calculation unit 112) of the distribution device 1 calculates, for example, for each program code stored in any of the queues 4, the evaluation result (hereinafter also referred to as the first evaluation result) by the evaluation device 3 for the program code previously created by the user who created each program code (hereinafter also referred to as the first program code) for the problem corresponding to each program code.

[0044] Specifically, the time calculation unit 112 calculates, for example, for each program code stored in any of the queues 4, the time required to evaluate (execute) the first program code corresponding to each program code (hereinafter also referred to as the first time) as the first evaluation result.

[0045] Furthermore, the time calculation unit 112 calculates, for example, for each program code stored in any of the queues 4, the evaluation result (hereinafter also referred to as the second evaluation result) obtained by the evaluation device 3 for a program code (hereinafter also referred to as the second program code) that was previously created using the programming language corresponding to each program code for a problem corresponding to each program code.

[0046] Specifically, the time calculation unit 112 calculates, for example, for each program code stored in one of the queues 4, the time required to evaluate (execute) the second program code corresponding to each program code (hereinafter also referred to as the second time) as the second evaluation result.

[0047] The time calculation unit 112 then calculates, for example, for each program code stored in one of the queues 4, the longer of the first time and the second time corresponding to each program code, as the predicted time required to evaluate (execute) each program code (hereinafter also simply referred to as the predicted time).

[0048] Furthermore, the time calculation unit 112 calculates, for example, the total predicted time corresponding to the program code stored in each queue 4 (hereinafter also referred to as the first predicted waiting time) for each queue 4. That is, the time calculation unit 112 calculates, for example, the predicted waiting time when the program code received by the code reception unit 111 is stored in each queue 4 as the first predicted waiting time.

[0049] Furthermore, the time calculation unit 112 calculates, for example, for each program code stored in one of the queues 4, the sum of the predicted time corresponding to each program code and the predicted time corresponding to each of the one or more program codes stored before each program code (hereinafter also referred to as the second predicted waiting time). In other words, the time calculation unit 112 calculates the predicted waiting time for each program code currently stored in each queue 4 as the second predicted waiting time.

[0050] Furthermore, the time calculation unit 112 identifies, for example, the storage order (arrival order) of each program code stored in one of the queues 4.

[0051] The queue identification unit 113 of the distribution device 1 identifies the queue 4 (hereinafter also referred to as the first queue 4) that stores the new program code received by the code reception unit 111, based on, for example, the first evaluation result and the second evaluation result for each program code stored in one of the queues 4.

[0052] Specifically, the queue identification unit 113 identifies the queue 4 with the shortest first predicted waiting time calculated by the time calculation unit 112 as the queue 4 (first queue 4) to store the program code received by the code reception unit 111.

[0053] The code storage unit 114 of the distribution device 1 stores, for example, the new program code (hereinafter also referred to as the specific program code) received by the code reception unit 111 for the queue 4 identified by the queue identification unit 113.

[0054] Next, we will describe the outline of the information notification process in the first embodiment.

[0055] As shown in Figure 7, the information acquisition unit 211 of the notification device 2 acquires information from each of the evaluation devices 3, including, for example, the progress of the evaluation of the program code being evaluated (hereinafter also referred to as evaluation progress information). Evaluation progress information is, for example, information transmitted from the evaluation device 3 each time an evaluation using each test data is completed when the evaluation of each program code is performed using multiple test data.

[0056] Furthermore, the information acquisition unit 211 acquires, for example, the second predicted waiting time calculated by the distribution device 1 (time calculation unit 112). Specifically, the information acquisition unit 211 receives, for example, the second predicted waiting time notified by the distribution device 1.

[0057] The information generation unit 212 of the notification device 2 generates information including the second predicted waiting time acquired by the information acquisition unit 211 (hereinafter also referred to as predicted waiting time information) for each program code stored in any of the queues 4.

[0058] The information notification unit 213 of the notification device 2 notifies the notification destination corresponding to the user who generated each program code of the evaluation progress information acquired by the information acquisition unit 211 for each program code being evaluated in each of the evaluation devices 3.

[0059] Furthermore, the information notification unit 213 notifies the notification destination corresponding to the user who generated each program code, for example, the predicted waiting time information generated by the information generation unit 212 for each program code stored in any of the queues 4.

[0060] In other words, the sorting device 1 in this embodiment predicts the waiting time for the evaluation of each program code stored in the queue 4 by using the evaluation results for each user of the program code created for each problem and the evaluation results for each programming language of the program code created for each problem.

[0061] As a result, the distribution device 1 in this embodiment can accurately predict the waiting time for evaluation of each program code stored in queue 4. Therefore, for example, when the distribution device 1 receives a new program code, it stores the received new program code in the queue 4 with the shortest waiting time. This may cause This makes it possible to do so. Therefore, the distribution device 1 has a waiting time for evaluation of each program code stored in queue 4. but shortening This may cause It becomes possible to do so.

[0062] In particular, in the information processing system 10 of this embodiment, a queue 4 is provided for each evaluation device 3, so each queue 4 stores only the program code that the evaluation device 3 corresponding to that queue 4 will evaluate. Therefore, in the information processing system 10 of this embodiment, it is possible to consider that the evaluation device 3 that performs the evaluation for each program code is determined at the time each program code is stored in the queue 4. Furthermore, it is possible to determine that the only program codes that affect the waiting time for evaluation of each program code are other program codes stored in the same queue 4 as each program code.

[0063] Therefore, in this embodiment, when the distribution device 1 predicts the waiting time corresponding to a specific program code stored in a particular queue 4 among the queues 4, it is possible to accurately predict the waiting time corresponding to a specific program code by using only the predicted time required to evaluate each program code stored in the particular queue 4. Accordingly, in this embodiment, the distribution device 1 can suppress the processing load for predicting the waiting time corresponding to each program code and the processing load for notifying the notification device 2 of changes in the waiting time prediction results, compared to, for example, the case where each program code is stored in a separate queue.

[0064] Furthermore, the notification device 2 in this embodiment, for example, periodically transmits evaluation progress information and predicted waiting time information corresponding to each program code currently stored in each queue 4 to the user who created each program code.

[0065] As a result, the notification device 2 in this embodiment can reduce the frequency of occurrences such as inquiries to confirm whether or not the evaluation of each program code has been completed. Therefore, the notification device 2 in this embodiment can improve convenience for each user and reduce the processing load on the distribution device 1 associated with inquiries from each user.

[0066] [Details of the first embodiment] Next, the details of the code distribution process in the first embodiment will be described. Figure 8 is a flowchart illustrating the details of the code distribution process in the first embodiment. Figures 9 to 11 are flowcharts illustrating the details of the information notification process in the first embodiment. Furthermore, Figure 12 is a diagram illustrating the details of the information notification process in the first embodiment.

[0067] [Code distribution process] First, the code distribution process in the first embodiment will be described.

[0068] As shown in Figure 8, the code receiving unit 111 waits until it receives a new program code, for example, from the user terminal 5 (NO in S11).

[0069] Then, when a new program code is received (YES in S11), the time calculation unit 112 calculates, for example, the time (first time) required to evaluate the program code (first program code) previously created by the user who created each program code for the problem corresponding to each program code, for each program code stored in any of the queues 4 (S12). In this case, the time calculation unit 112 also calculates, for example, the time (second time) required to evaluate the program code (second program code) previously created using the programming language corresponding to each program code for the problem corresponding to each program code, for each program code stored in any of the queues 4 (S12).

[0070] Specifically, the time calculation unit 112 may calculate, for example, the maximum or average time required to execute the first program code for each program code stored in any of the queues 4 as the first time. Alternatively, the time calculation unit 112 may calculate, for example, the maximum or average time required to execute the second program code for each program code stored in any of the queues 4 as the second time.

[0071] Furthermore, the time calculation unit 112 may calculate, for example, the time required to execute the first program code created immediately before by the user for each program code stored in any of the queues 4 as the first time. Also, the time calculation unit 112 may calculate, for example, the time required to execute the second program code created immediately before by the user for each program code stored in any of the queues 4 as the second time.

[0072] More specifically, the time calculation unit 112 may, for example, calculate the maximum time required to execute the first program code as the first time for each program code stored in any of the queues 4, and calculate the average time required to execute the second program code as the second time.

[0073] Next, the time calculation unit 112 calculates, for example, for each program code stored in one of the queues 4, the longer of a first time and a second time corresponding to each program code as the predicted time required to evaluate each program code (S13).

[0074] This prevents the distribution device 1 from underestimating the predicted time for each program code stored in queue 4. Therefore, as will be described later, the distribution device 1 can prevent the waiting time for each program code stored in queue 4 from being underestimated.

[0075] Furthermore, the time calculation unit 112 calculates, for example, the total predicted time (first predicted waiting time) corresponding to the program code stored in each queue 4 for each queue 4 (S14).

[0076] In other words, the time calculation unit 112 calculates, for example, the first predicted waiting time for each queue 4 when the new program code received in the processing of S11 is stored therein.

[0077] In addition, during the processing of S14, the time calculation unit 112 calculates, for example, for each program code stored in any of the queues 4, the sum of the predicted time corresponding to each program code and the predicted time corresponding to each of the one or more program codes stored before each code (second predicted waiting time).

[0078] Subsequently, the queue identification unit 113 identifies the queue 4 with the shortest first predicted waiting time calculated in the S14 process as the queue 4 (first queue 4) to store the program code received by the code reception unit 111 (S15).

[0079] Then, the code storage unit 114 stores the program code received by the code reception unit 111 in the queue 4 identified in the processing of S15, for example (S16).

[0080] As a result, the distribution device 1 stores the new program code received in the S11 process in the queue 4 with the shortest waiting time. This may cause It becomes possible to do so.

[0081] The time calculation unit 112 may, for example, perform the S14 process each time a program code stored in queue 4 is retrieved (each time the evaluation of the program code stored in queue 4 is completed in the evaluation device 3).

[0082] Specifically, in this case, the time calculation unit 112 identifies, for example, the first predicted waiting time corresponding to the queue 4 from which the program code was retrieved, from the first predicted waiting time calculated in the previous S14 process. Then, the time calculation unit 112 recalculates (updates) the first predicted waiting time corresponding to the queue 4 from which the program code was retrieved by subtracting, for example, the predicted time corresponding to the program code retrieved from the queue 4 from the identified first predicted waiting time. Similarly, in this case, the time calculation unit 112 identifies, for example, each of the second predicted waiting times corresponding to each program code stored in the queue 4 from which the program code was retrieved, from the second predicted waiting time calculated in the previous S14 process. Then, the time calculation unit 112 recalculates (updates) the second predicted waiting time corresponding to each program code stored in the queue 4 from which the program code was retrieved by subtracting, for example, the predicted time corresponding to the program code retrieved from the queue 4 from each of the identified second predicted waiting times.

[0083] Furthermore, the time calculation unit 112 may, for example, perform the processing of S12 to S14 for a new program code after the processing of S16 has been performed.

[0084] Specifically, in this case, the time calculation unit 112 calculates the predicted time corresponding to the new program code by performing the S12 and S13 processes for the new program code after the S16 process has been completed. Then, the time calculation unit 112 identifies the first predicted waiting time corresponding to the queue 4 where the new program code was stored from among the first predicted waiting times calculated in the previous S14 process. Subsequently, the time calculation unit 112 calculates the first predicted waiting time corresponding to the queue 4 where the new program code was stored by adding the predicted time calculated for the new program code to the identified first predicted waiting time.

[0085] Furthermore, if the code reception unit 111 receives a new program code, the time calculation unit 112 may, for example, skip the processing from S12 to S14 and use the first predicted waiting time calculated in the previous processing of S14 to perform the processing from S15 onward.

[0086] As a result, the sorting device 1 can, for example, determine the time required to execute the code sorting process. but shortening This may cause It becomes possible to do so.

[0087] [Information notification processing] Next, the information notification process in the first embodiment will be described.

[0088] As shown in Figure 9, the information generation unit 212 waits, for example, until the first notification timing occurs (NO in S21). The first notification timing may be, for example, the timing when the code receiving unit 111 of the distribution device 1 receives a new program. Alternatively, the first notification timing may be, for example, every predetermined time elapsed since the timing when the code receiving unit 111 of the distribution device 1 receives a new program. Furthermore, the first notification timing may be, for example, the timing when a program code is retrieved from the queue 4 (i.e., the timing when the evaluation of the program code is completed in the evaluation device 3). In addition, the first notification timing may be, for example, the timing when the new program code received by the code receiving unit 111 of the distribution device 1 is stored in the queue 4.

[0089] Then, when the first notification timing is reached (YES in S21), the information generation unit 212 generates information including the storage order identified by the distribution device 1 (hereinafter also referred to as storage order information) for each program code stored in any of the queues 4 (S22).

[0090] Specifically, the information acquisition unit 211 receives, for example, the storage order of each program code stored in one of the queues 4 from the distribution device 1. Then, the information generation unit 212 generates, for example, storage order information for each program code stored in one of the queues 4, including the storage order received from the distribution device 1.

[0091] Subsequently, the information notification unit 213 notifies the notification destination of the storage order information generated in the process of S22 for each program code stored in one of the queues 4 (S23).

[0092] Specifically, the information notification unit 213 sends, for example, the storage order information generated in the S22 process to the notification destination (for example, the user terminal 5) corresponding to the user who created each program code, for each program code stored in one of the queues 4.

[0093] Furthermore, as shown in Figure 10, the information generation unit 212 waits, for example, until the second notification timing occurs (NO in S31). The second notification timing may be, for example, the timing when the code receiving unit 111 of the distribution device 1 receives a new program, similar to the first notification timing. Alternatively, the second notification timing may be, for example, the timing each predetermined time interval after the timing when the code receiving unit 111 of the distribution device 1 receives a new program, similar to the first notification timing. Alternatively, the second notification timing may be, for example, the timing when a program code is retrieved from the queue 4 (i.e., the timing when the evaluation of the program code is completed in the evaluation device 3), similar to the first notification timing. Furthermore, the second notification timing may be, for example, the timing when the new program code received by the code receiving unit 111 of the distribution device 1 is stored in the queue 4, similar to the first notification timing.

[0094] Then, when the second notification timing is reached (YES in S31), the information generation unit 212 generates waiting time information, including the second predicted waiting time calculated by the distribution device 1, for each program code stored in any of the queues 4 (S32).

[0095] Specifically, the information acquisition unit 211 receives, for example, the second predicted waiting time for each program code stored in one of the queues 4 from the distribution device 1. Then, the information generation unit 212 generates, for example, predicted waiting time information for each program code stored in one of the queues 4, including the second predicted waiting time received from the distribution device 1.

[0096] Then, the information notification unit 213 notifies the notification destination of the predicted waiting time information generated in the processing of S32 for each program code stored in one of the queues 4 (S33).

[0097] Specifically, the information notification unit 213 sends, for example, the predicted waiting time information generated in the S32 process to the notification destination (for example, the user terminal 5) corresponding to the user who created each program code, for each program code stored in one of the queues 4.

[0098] Furthermore, the information generation unit 212 waits, for example, until it receives evaluation progress information about the program code being evaluated from the evaluation device 3 (NO in S41).

[0099] Then, when evaluation progress information is received from the evaluation device 3 (YES in S41), the information generation unit 212 generates information including the received evaluation progress information (S42).

[0100] Subsequently, the information notification unit 213 notifies the recipient of the information generated in the process of S42, for example (S43).

[0101] Specifically, the information notification unit 213 sends, for example, the information generated in the S42 process to a notification destination (for example, user terminal 5) corresponding to the user who created the program code corresponding to the received evaluation progress information.

[0102] Furthermore, for example, if any of the evaluation devices 3 acquire a program code stored in queue 4, or if any of the evaluation devices 3 transmit information indicating that the evaluation of a program code stored in queue 4 has started, the information generation unit 212 may generate information indicating that the evaluation of that program code has started (hereinafter also referred to as evaluation start information). The information notification unit 213 may then, for example, transmit the generated evaluation start information to the notification recipient.

[0103] Furthermore, for example, if the evaluation result of a program code is received from one of the evaluation devices 3, the information generation unit 212 may generate information including the evaluation result of that program code (hereinafter also referred to as evaluation completion information). The information notification unit 213 may then, for example, transmit the generated evaluation completion information to the notification recipient.

[0104] Furthermore, the notification device 2 may, for example, perform information notification processing only for some of the program codes among the program codes stored in one of the queues 4.

[0105] Furthermore, the notification device 2 (information notification unit 213) may process the storage order information generated in the S22 process and the waiting time information generated in the S32 process into information that can be visually understood by the user before transmitting it. In other words, the notification device 2 may transmit information indicating the storage status of each program code in queue 4. Specific examples of the processed storage order information and waiting time information will be described below.

[0106] [Storage order information and waiting time information after processing] Figure 12 illustrates a specific example of displaying the storage order information and waiting time information after processing on the screen SC of a user terminal 5 or the like.

[0107] The screen SC shown in Figure 12 schematically displays queue 4a, which stores program codes in the order they arrive. Specifically, the example shown in Figure 12 shows a state in which eight program codes sent by the user are stored in queue 4a. That is, in the example shown in Figure 12, each of the eight program codes is stored in storage areas 4a1, 4a2, 4a3, 4a4, 4a5, 4a6, 4a7, and 4a8 in queue 4a.

[0108] Furthermore, in the example shown in Figure 12, the program code stored in the shaded storage area 4a4, among the eight program codes stored in queue 4a, is the program code corresponding to the user to whom the information will be notified. In addition, in the example shown in Figure 12, the additional information IF4 is displayed indicating that the storage order of the program code stored in the shaded storage area 4a4 in queue 4a is "4th", and the time required until feedback of the evaluation result is received (second predicted waiting time) is "15 seconds".

[0109] Furthermore, in the example shown in Figure 12, additional information IF1, IF2, IF3, IF5, IF6, IF7, and IF8 are displayed as information corresponding to storage areas 4a1, 4a2, 4a3, 4a5, 4a6, 4a7, and 4a8, respectively. In other words, the screen SC shown in Figure 12 also displays information about program codes corresponding to users other than the user to whom the information is notified.

[0110] In the example shown in Figure 12, we have described the case where the waiting time information of the program code stored in one queue 4, queue 4a, is displayed. However, it is also possible to display the waiting time information of the program code stored in each of the multiple queues 4 (for example, queues 4a, 4b, and 4c) separately for each queue 4, or to display the waiting time information of the program code stored in each of the multiple queues 4 merged into one queue 4 (for example, queue 4a).

[0111] As described above, when the distribution device 1 in this embodiment receives a new program code, it calculates, for each program code stored in one of the multiple queues 4, a first evaluation result from multiple evaluation devices 3 for the first program code previously created by the user who created each program code for the problem corresponding to each program code, and a second evaluation result from multiple evaluation devices 3 for the second program code previously created using the programming language corresponding to each program code for the problem corresponding to each program code. Then, based on the first evaluation result and the second evaluation for each program code stored in one of the multiple queues 4, the distribution device 1 identifies a first queue 4 to store the new program code from among the multiple queues 4. After that, the distribution device 1 stores the new program code in the identified first queue 4.

[0112] Furthermore, the notification device 2 in this embodiment notifies the notification destination corresponding to the user who generated the new program code of the information about the new program code stored in the identified first queue 4.

[0113] As a result, the distribution device 1 in this embodiment can accurately predict the evaluation time for each program code stored in queue 4. Therefore, when the distribution device 1 receives a new program code, it stores the received program code in the queue 4 with the shortest waiting time. This may cause This makes it possible to reduce the time (waiting time) required for evaluation of each program code. but shortening This may cause It becomes possible to do so.

[0114] Furthermore, the notification device 2 in this embodiment can suppress the frequency of occurrences such as inquiries to confirm whether or not the evaluation of each program code has been completed. Therefore, the notification device 2 in this embodiment can improve convenience for each user and reduce the processing load on the distribution device 1 associated with inquiries from each user. [Explanation of Symbols]

[0115] 1: Sorting device 2: Notification device 3: Evaluation device 4: Queue 5: User terminal 10: Information Processing Systems 101:CPU 102: Memory 103: Communication Interface 104:Storage medium 105: Bus 111: Code Reception Department 112: Time calculation unit 113: Queue Identification Department 114: Code storage unit 201:CPU 202: Memory 203: Communication Interface 204:Storage medium 205: Bus 211: Information acquisition department 212: Time calculation unit 213: Information generation section 214: Information Notification Department 301:CPU 302: Memory 303: Communication Interface 304:Storage medium 305: Bus

Claims

1. Multiple evaluation devices that each evaluate the program code created by the user, A plurality of queues corresponding to each of the plurality of evaluation devices, A distribution device that distributes the program code to one of the plurality of queues, An information processing system comprising: a notification device that notifies the source of the program code of information relating to the program code stored in any of the plurality of queues, The aforementioned distribution device is When a new program code is received, for each program code stored in any of the multiple queues, a first evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for a first program code previously created by the user who created the program code for the problem corresponding to the program code, and indicates the first time required to evaluate the first program code; and a second evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for a second program code previously created using the programming language corresponding to the program code for the problem corresponding to the program code, and indicates the second time required to evaluate the second program code. For each program code stored in any of the aforementioned queues, the longer of the first time and the second time corresponding to each program code is calculated as the estimated time required for the execution of each program code. For each of the aforementioned queues, the total estimated time corresponding to the program code stored in each queue is calculated. The queue with the shortest total time among the plurality of queues is identified as the first queue for storing the new program code. The new program code is stored in the identified first queue, The notification device notifies the notification destination corresponding to the user who generated the new program code of the information stored in the identified first queue. An information processing system characterized by the following:

2. In claim 1, The aforementioned distribution device is For each program code stored in the first queue, the first evaluation result corresponding to each program code and the second evaluation result corresponding to each program code are calculated. Based on the first evaluation result and the second evaluation result for each program code stored in the first queue, information about the new program code is generated. An information processing system characterized by the following:

3. In claim 1, The first time is the maximum or average time required to execute the first program code. The second time is the maximum or average time taken to execute the second program code. An information processing system characterized by the following:

4. In claim 1, The aforementioned distribution device is For each program code stored in the first queue, the longer of the first time and the second time corresponding to each program code is calculated as the estimated time required to execute each program code. The total time is calculated by determining the predicted time corresponding to the new program code and the predicted time corresponding to each of the one or more program codes stored before the new program code. The information including the calculated total time is notified as information regarding the new program code. An information processing system characterized by the following:

5. In claim 1, The notification device notifies the number of program codes stored before the new program code in the first queue as information about the new program code. An information processing system characterized by the following:

6. In claim 1, When the evaluation of the new program code using multiple test data is performed in a first evaluation device corresponding to the first queue among the multiple evaluation devices, the notification device notifies, as information relating to the new program code, that the evaluation of the new program code using specific test data included in the multiple test data has been completed in the first evaluation device. An information processing system characterized by the following:

7. In claim 1, The notification device notifies the notification recipient of information indicating the storage status of each program code in the first queue. An information processing system characterized by the following:

8. An information processing method in an information processing system comprising: a plurality of evaluation devices that each evaluate program code created by a user; a plurality of queues corresponding to each of the plurality of evaluation devices; a distribution device that distributes the program code to one of the plurality of queues; and a notification device that notifies the source of the program code of information about the program code stored in one of the plurality of queues, The aforementioned distribution device, When a new program code is received, for each program code stored in any of the multiple queues, a first evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for a first program code previously created by the user who created the program code for the problem corresponding to the program code, and indicates the first time required to evaluate the first program code; and a second evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for a second program code previously created using the programming language corresponding to the program code for the problem corresponding to the program code, and indicates the second time required to evaluate the second program code. For each program code stored in any of the aforementioned queues, the longer of the first time and the second time corresponding to each program code is calculated as the estimated time required for the execution of each program code. For each of the aforementioned queues, the total estimated time corresponding to the program code stored in each queue is calculated. The queue with the shortest total time among the plurality of queues is identified as the first queue for storing the new program code. The new program code is stored in the identified first queue, The notification device notifies the notification destination corresponding to the user who generated the new program code of the information about the new program code stored in the identified first queue. An information processing method characterized by the following:

9. When a new program code is received, a result calculation unit calculates, for each program code stored in one of a plurality of queues corresponding to each of a plurality of evaluation devices that evaluate program codes created by a user, a first evaluation result which indicates the evaluation result by the plurality of evaluation devices for a first program code previously created by the user who created each program code for the problems corresponding to each program code, and a second evaluation result which indicates the evaluation result by the plurality of evaluation devices for a second program code previously created using the programming language corresponding to for the problems corresponding to each program code, For each program code stored in any of the aforementioned queues, the longer of the first time and the second time corresponding to each program code is calculated as the estimated time required for the execution of each program code. A queue identification unit calculates the total estimated time corresponding to the program code stored in each of the plurality of queues, and identifies the queue with the shortest total time among the plurality of queues as the first queue for storing the new program code. The system includes a code storage unit that stores the new program code for the identified first queue, A sorting device characterized by the following features.

10. A result calculation unit calculates, for each program code stored in a first queue, which is included in a plurality of queues corresponding to each of a plurality of evaluation devices that each evaluate the program code created by the user, a first evaluation result which indicates the evaluation result by the plurality of evaluation devices for the first program code previously created by the user who created each program code for the problem corresponding to each program code, and which indicates the evaluation result by the plurality of evaluation devices for the second program code previously created using the programming language corresponding to each program code for the problem corresponding to each program code, and which indicates the evaluation result by the plurality of evaluation devices for the second previously created using the programming language corresponding to each program code, and which indicates the evaluation result by the plurality of evaluation devices for the second program code for the problem corresponding to each program code, An information generation unit generates information about a specific program code among the program codes stored in the first queue based on the first evaluation result and the second evaluation result for each program code stored in the first queue, The system includes an information notification unit that notifies a notification destination corresponding to the user who generated the specific program code, of the information related to the generated specific program code. A notification device characterized by the following features.

11. When a new program code is received, for each program code stored in one of the queues corresponding to each of the multiple evaluation devices that evaluate the program code created by the user, a first evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for the first program code previously created by the user who created the program code for the problem corresponding to the program code, and indicates the first time required to evaluate the first program code; and a second evaluation result is calculated, which is the evaluation result by the multiple evaluation devices for the second program code previously created using the programming language corresponding to the program code for the problem corresponding to the program code, and indicates the second time required to evaluate the second program code; For each program code stored in any of the aforementioned queues, the longer of the first time and the second time corresponding to each program code is calculated as the estimated time required for the execution of each program code. For each of the aforementioned queues, the total estimated time corresponding to the program code stored in each queue is calculated. The queue with the shortest total time among the plurality of queues is identified as the first queue for storing the new program code. The new program code is stored in the identified first queue. A sorting program characterized by having a computer perform the processing.

12. For each program code stored in a first queue, which is included in a plurality of queues corresponding to each of a plurality of evaluation devices that each evaluate the program code created by the user, a first evaluation result is calculated, which is the evaluation result by the plurality of evaluation devices for the first program code previously created by the user who created each program code for the problem corresponding to each program code, and indicates the first time required to evaluate the first program code; and a second evaluation result is calculated, which is the evaluation result by the plurality of evaluation devices for the second program code previously created using the programming language corresponding to each program code for the problem corresponding to each program code, and indicates the second time required to evaluate the second program code; Based on the first evaluation result and the second evaluation result for each program code stored in the first queue, information about a specific program code among the program codes stored in the first queue is generated. Information regarding the generated specific program code is notified to the notification destination corresponding to the user who generated the specific program code. A notification program characterized by having a computer perform a process.

Images