Ship assignment plan drafting support system and ship assignment plan drafting support method

Abstract

To support the planning of a ship assignment plan.SOLUTION: A state St is a state based on coal yard information including a decrease amount of coal in each of the coal yards, coal contract information including an available supply amount of each of one or more coal suppliers and the number of necessary voyage days of the transport ship to the coal yard, ship information including a load amount and an available unloading date of each of the transport ships, and ship operation information including information indicating a daily operation status of each of the transport ships; A model for generating a ship assignment plan of a transport ship is generated by performing deep reinforcement learning for obtaining an action value Q (S, a) by giving a reward rt corresponding to a state St + 1 after performing an action at with each operation situation of the transport ship as the action at, and each ship assignment plan of the transport ship is generated by inputting an initial value of the state St to the model.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a ship allocation plan creation support system and a ship allocation plan creation support method. [Background technology]

[0002] Patent Document 1 describes a ship allocation plan creation device that is designed to optimally determine the order of port calls, number of voyages, and timing of cargo handling when delivering a predetermined amount of cargo by multiple ships. The ship allocation plan creation device determines the order of port calls and number of voyages from the loading and unloading ports and transport volume of the transported goods, taking into account time-independent constraints including at least the ship's load capacity and the number of ships, and creates a ship allocation plan from the determined order of port calls and number of voyages.

[0003] Patent Document 2 describes an inference device for raw material transport ship allocation planning, which is configured for the purpose of creating an efficient berth unloading plan and realizing a stable supply of raw materials. The inference device for raw material transport ship allocation planning determines transport timing using a ship allocation planning device, using a suitable ship selection knowledge base for determining transport ships suitable for realizing the annual transport plan, a loading decision knowledge base for realizing the load factor of transport ships, etc., and formulates a long-term raw material transport ship allocation plan, while responding to fluctuations in raw material ship movements, etc., performing daily operations using a ship arrival adjustment device and flexibly changing the plan.

[0004] Non-Patent Document 1 describes DQN (Deep Q-Network), which is one of the deep reinforcement learning methods. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-1114 [Patent Document 2] Japanese Patent Application Publication No. 8-272402 [Non-patent literature]

[0006] [Non-Patent Document 1] “Deep Learning Technology Deep Reinforcement Learning”, [online], Toshiba Digital Solutions Corporation, [Retrieved July 29, 2024], Internet,<URL:https: / / www.global.toshiba / jp / company / digitalsolution / articles / sat / 1804_1.html> Summary of the Invention [Problem to be solved by the invention]

[0007] In the operation of coal-fired power plants, in order to ensure stable output at all times, it is necessary to properly maintain and manage the remaining coal in each coal yard attached to each power plant.For this reason, the site in charge of managing each coal yard creates a shipping plan (Gantt chart (operation control sheet)) for transport ships and plans the shipping plan.

[0008] However, when creating a shipping schedule, it is necessary to take into account various constraints, various factors, and various events (for example, the operating status of shipping ships due to weather conditions, etc.). Furthermore, coal is supplied to coal yards by shipping ships from multiple suppliers, and it is also necessary to manage the quantity and quality of coal available from each supplier. As such, creating a shipping schedule requires consideration of complex conditions, which requires effort and expertise, and reducing costs (human costs, time costs, etc.) and securing the necessary personnel are issues.

[0009] Furthermore, Patent Documents 1 and 2 do not fully consider the constraints and matters described above, and are therefore unable to create shipping plans with practical accuracy.

[0010] The present invention has been made in consideration of this background, and aims to provide a ship allocation plan development support system and a ship allocation plan development support method that can support work related to the development of ship allocation plans. [Means for solving the problem]

[0011] One of the present inventions for achieving the above-mentioned object is a system (shipping plan planning support system) that supports the creation of coal transport ship routing plans, which is configured using an information processing device having a processor and a storage device, and has one or more coal stockpiles where coal is to be unloaded as an environment, one or more transport ship dispatchers as agents, and a state St based on coal stockpile information including the amount of coal that has been reduced at each of the coal stockpiles, coal contract information including the amount of coal that can be supplied from each of one or more coal suppliers and the number of days the transport ship needs to sail to the coal stockpiles, ship information including the load capacity of each of the transport ships and the date on which unloading is possible, and ship operation information including information showing the daily operating status of each of the transport ships, and the operation status of each of the transport ships is set as an action at, and a reward rt corresponding to the state St+1 after the action at is performed is assigned, and deep reinforcement learning is performed to determine an action value Q(S, a) to generate a model for generating a shipping plan for the transport ships, and the initial value of the state St is input into the model to generate each of the shipping plan for the transport ships.

[0012] Other problems and solutions disclosed in the present application will be made clear in the detailed description and drawings. [Effects of the Invention]

[0013] According to the present invention, it is possible to assist in the formulation of a shipping plan. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a planning support system. [Figure 2] FIG. 1 is a diagram illustrating an overview of deep reinforcement learning by a planning support system. [Figure 3] FIG. 2 is a diagram illustrating main functions of the planning support device. [Figure 4A] 10 is an example of coal yard information. [Figure 4B] 1 is an example of coal contract information. [Figure 4C] 10 is an example of transport ship information. [Figure 4D] 10 is an example of transport ship operation information. [Figure 5] This is an example of a shipping plan (Gantt chart). [Figure 6] 10 is a flowchart illustrating a model learning process. [Figure 7A] FIG. 10 is a schematic diagram illustrating an example of how to give a reward rt. [Figure 7B] FIG. 10 is a schematic diagram illustrating an example of how to give a reward rt. [Figure 7C] FIG. 10 is a schematic diagram illustrating an example of how to give a reward rt. [Figure 7D] FIG. 10 is a schematic diagram illustrating an example of how to give a reward rt. [Figure 8] FIG. 10 is a diagram showing examples of an evaluation index, an action at, an action value Q(s, a), and a reward rt for each state St. [Figure 9] FIG. 10 is a diagram illustrating an example of the process or work flow leading up to the creation of a shipping plan. [Figure 10] 1 is an example of an information processing device used to realize a planning support system. DETAILED DESCRIPTION OF THE INVENTION

[0015] An embodiment of the present invention will now be described with reference to the accompanying drawings. In the following description, the letter "S" before a reference numeral denotes a processing step.

[0016] FIG. 1 shows a schematic configuration of a planning support system 1 (hereinafter referred to as "planning support system 1"), which is an information processing system that supports work related to planning a shipping plan, and which will be explained as one embodiment of the present invention.

[0017] As shown in the figure, the planning support system 1 includes a ship allocation plan planning support device (hereinafter referred to as "planning support device 100"), a coal stockpile monitoring device 200, and a ship allocation management device 300. All of these are configured using information processing devices (computers), and are connected to each other via a communication network 5 so that they can communicate with each other bidirectionally. The communication network 5 is a wired or wireless communication infrastructure, such as the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a dedicated line, or various public communication networks.

[0018] The planning support device 100 supports the planning of coal transport ship (coal carrier) allocation plans, which are created for the purpose of ensuring that the remaining amount of coal (hereinafter referred to as "remaining coal amount") required for stable operation of one or more coal-fired power plants is stably maintained in a coal yard.

[0019] The coal stockpile monitoring device 200 provides the planning support device 100 with various information (remaining amount of coal, amount of coal decrease, etc.) used by the planning support device 100 when planning a shipping plan via the communication network 5 as needed.

[0020] The ship allocation management device 300 provides the planning support device 100 with various information (such as the operating status of each transport ship and the amount of coal carried) used by the planning support device 100 to plan a ship allocation plan via the communication network 5 at any time.

[0021] The planning support device 100 generates (learns) a shipping schedule planning model (hereinafter also referred to as "model"), which is a DQN model, by performing deep reinforcement learning using DQN (Deep-Q-Network) taking into consideration various constraints, various matters, various events, etc. Then, the planning support device 100 generates a shipping schedule (Gantt chart (operation control table), etc.) using the generated model.

[0022] When generating the model, the planning support device 100 takes into consideration, as constraints, the upper and lower limits of the remaining coal amount in each of one or more coal stockpiles, the coal contract amount (supplyable amount, procurable coal amount) between the operator of the coal stockpiles and each of one or more coal suppliers (Australia, Indonesia, the United States, Canada, South Africa, etc.), etc.

[0023] In addition, the planning support device 100 takes into consideration various factors when generating a model, such as the number of transport days for each supplier (hereinafter referred to as "required transport days"), the remaining amount of coal at each supplier (the remaining amount that can be procured from each supplier = the contracted amount minus the amount of coal that has already been supplied), the number of operable transport ships and the amount that each can transport, each transport ship's lay-up plan, whether or not a transport ship will be discharged to a second port, the frequency of transporting high-grade coal (such as from Australia), the operating status of transport ships, and predicted daily weather.

[0024] FIG. 2 is a diagram illustrating an outline of deep reinforcement learning by the planning support device 100.

[0025] As shown in the figure, when learning a model, the planning support device 100 sets each coal yard as an environment, which is one or more coal yards where coal is to be unloaded.

[0026] Furthermore, the planning support device 100 sets one or more transport ship operation commanders (such as the captain or the manager of the coal yard) as agents.

[0027] In addition, the planning support device 100 sets, as the state St in Q-learning, a state based on coal yard information including the amount of coal depleted in each coal yard, coal contract information including the amount of coal that can be supplied from each of one or more coal suppliers and the number of days required for the transport ship to travel to the coal yard, ship information including the loading capacity of each transport ship and the date on which it can be unloaded, and ship operation information including information showing the daily operating status of each transport ship.

[0028] Furthermore, the planning support device 100 sets the operation status of each transport ship (for example, at least one of demurrage, unloading, voyage, and unloading at two ports) as the action at in Q-learning.

[0029] Furthermore, the planning support device 100 sets the reward rt in Q-learning as one of the following (1) to (4) depending on the state St after performing a certain action at. (1) When the remaining amount of coal in the coal yard is within a predetermined optimum range, a higher reward rt is given than when it is outside the range. (2) The smaller the difference between the amount of coal lost in the coal yard and the amount of coal supplied from the supplier, the higher the reward rt will be. (3) The reward rt is given so that the more consecutive times coal whose quality is below a preset standard is unloaded, the lower the reward rt is. (4) The greater the deviation from the ideal rate of decrease in the supplier's available supply volume in a given period, the lower the reward rt is.

[0030] 3 shows the main functions of the planning support device 100. As shown in the figure, the planning support device 100 has the functions of a memory unit 110, an information acquisition management unit 125, a model learning unit 130, a shipping plan generation unit 135, and a shipping plan correction unit 140.

[0031] Of these, the storage unit 110 stores each piece of information (data) including model input information 111, remuneration information 112, a model 113, and a shipping plan 114.

[0032] The model input information 111 includes information used by the model learning unit 130 when learning the model 113 and information used when creating a shipping plan using the model 113.

[0033] The reward information 112 includes information about the reward rt that the model learning unit 130 gives when the model 113 undergoes deep reinforcement learning.

[0034] The model 113 generates (outputs) a shipping plan 114 based on the given model input information 111 .

[0035] The shipping plan 114 is a shipping plan generated by the model 113.

[0036] The information acquisition management unit 125 acquires the model input information 111 via the communication network 5 or a user interface, for example, and manages it in the storage unit 110 .

[0037] The model learning unit 130 learns the model 113 using the model input information 111. The model learning unit 130 is, for example, based on Non-Patent Document 1 ("Deep Learning Technology Deep Reinforcement Learning", [online], Toshiba Digital Solutions Corporation, [searched July 29, 2024], Internet,<URL:https: / / www.global.toshiba / jp / company / digitalsolution / articles / sat / 1804_1.html> ) and train the model 113 using the mechanism described in the previous section.

[0038] The shipping plan generation unit 135 inputs the model input information 111 into the model 113 as initial values, and manages the information output by the model 113 as a shipping plan 114 in the storage unit 110. In addition, the shipping plan generation unit 135 modifies the model 113, for example, through interactive processing with the user performed via software that modifies the model 113 (for example, adding logic to avoid cases where the remaining coal amount in a coal yard exceeds the upper or lower limit, or where a supplier with no remaining coal is selected).

[0039] The shipping plan correction unit 140 corrects the shipping plan 114 generated (output) by the model 113. The shipping plan correction unit 140 corrects (adjusts) the shipping plan 114, for example, through interactive processing with the user performed via software that corrects the shipping plan 114, or by applying automatic correction logic to the shipping plan. In addition, the shipping plan correction unit 140 corrects the model 113 (for example, by adding logic to avoid cases where the remaining coal amount in a coal yard exceeds an upper or lower limit, or where a supplier with no remaining coal is selected) through interactive processing with the user performed via software that corrects the model 113.

[0040] <Model input information> 4A is an example of coal yard information 1111, which is input information (model input information 111) of the model 113 used by the planning support device 100 when learning the model 113. The illustrated coal yard information 1111 includes upper and lower limit values ​​indicating the appropriate amount of remaining coal in each coal yard, and information indicating the daily coal decrease amount for each coal yard.

[0041] 4B is an example of coal contract information 1112, which is the model input information 111. The illustrated coal contract information 1112 includes, for each of the contractors (coal suppliers), information indicating the amount of coal currently contracted with each contractor (hereinafter referred to as the "contracted coal amount") and the number of days required to procure coal from each contractor by transport ship (hereinafter referred to as the "number of voyage days").

[0042] 4C is an example of transport ship information 1113, which is the model input information 111. The transport ship information 1113 shown includes information such as the affiliation of each transport ship (the coal storage yard of the supply destination), the amount of coal loaded, and the possible unloading date.

[0043] FIG. 4D is an example of transport ship operation information 1114, which is the model input information 111. The transport ship operation information 1114 shown includes information indicating the operation status of each transport ship on a daily basis. "S1" to "S10" written in the column direction of the table shown in the figure are identifiers for each transport ship. In the table, a "blank" indicates that the transport ship is in operation, and "99" indicates that the transport ship is out of operation.

[0044] 5 is an example of a vessel allocation plan (Gantt chart) 114. As shown in the figure, the illustrated Gantt chart includes information indicating the future operating status of each transport vessel for a predetermined period of time.

[0045] <Model training> 6 is a flowchart illustrating the process (hereinafter referred to as "model learning process S600") that the model learning unit 130 of the planning support device 100 performs when learning the model 113. The model learning process S600 will be described below with reference to this figure.

[0046] First, the model learning unit 130 acquires information used for learning the model 113, such as the model input information 111, from the storage unit 110 (S611).

[0047] Next, the model learning unit 130 inputs the initial value of the state St (for example, the current state St) into the model 113 to find the action value Q(St,at) (S612).

[0048] Next, the model learning unit 130 obtains the reward rt by performing the action at (for example, the action selected by the ε-greedy method) and stores the set of the state St, the action at, the reward rt, the state St+1, and the action value Q(St, at) (S613).

[0049] Next, the model learning unit 130 calculates the error (TD error) and updates the weight of the model 113 based on the calculated error (S614).

[0050] Next, the model learning unit 130 determines whether the processes of S612 to S614 have been performed a predetermined number of times (S615). If the processes of S612 to S614 have been performed a predetermined number of times (S615: YES), the model learning unit 130 ends the model learning process S600.

[0051] On the other hand, if the processes of S612 to S614 have not yet been performed the predetermined number of times (S615: NO), the process proceeds to S616.

[0052] In S616, the model learning unit 130 determines whether it is time to update the target network. If it is not time to update the target network (S616: NO), the process returns to S611. If it is time to update the target network (S616: YES), the process proceeds to S617.

[0053] In S617, the model learning unit 130 acquires an arbitrary number of past states St, actions at, rewards rt, states St+1, and action values ​​Q(St, at), and updates the weights of the target network with the set number of mini-batches.

[0054] Next, the model learning unit 130 synchronizes the weights of the target network with the model 113 (S618), after which the process returns to S612.

[0055] Incidentally, when training the model 113, it is necessary to consider the allocation of coal to be supplied to each coal yard when a transport ship unloads at two ports. For example, when a transport ship unloads at a first coal yard and then at a second coal yard, the amount of coal to be unloaded at the second coal yard is set taking into account the amount of coal to be unloaded at the first coal yard.

[0056] <How to give reward RT> Figure 7A is a schematic diagram illustrating an example of how to award the reward rt according to (1) above (a reward rt is awarded so that it is higher when the remaining coal in the coal yard is within a predetermined appropriate range than when it is outside that range).

[0057] If an absolute value is used as the value for evaluating the state St (hereinafter referred to as the "evaluation index"), it will not be possible to respond to changes in the settings of the upper and lower limits. In addition, it will be necessary to match the scale with other evaluation indexes. Therefore, when giving a reward rt using the method (1) described above, for example, as shown in the same figure, the reward rt will be given according to the value of the difference (degree of deviation) from the intermediate value of the upper and lower limits.

[0058] Figure 7B is a schematic diagram explaining an example of how to assign the reward rt according to (2) above (the smaller the difference between the amount of coal lost in the coal yard and the amount of coal supplied from the supplier, the higher the reward rt is assigned).

[0059] When awarding reward RT using the method (2) described above, for example, as shown in the figure, the difference between the average value of the daily coal reduction (kt / day) from the present to 30 days from now is calculated, and the reward RT is determined by a normalized value (the closer the supply capacity is to the coal reduction amount, the higher the reward RT is given).

[0060] FIG. 7C is a schematic diagram explaining how the reward rt is awarded according to (3) above (the reward rt is awarded so that the more consecutive times coal whose quality is below a preset standard value is unloaded, the lower the reward rt is).

[0061] When the reward rt is given using the method (3) described above, for example, as shown in the figure, the number of consecutive times low-grade coal (coal whose quality is below a specified standard) is counted, and the reward rt given when the number of times exceeds a specified number (for example, three times) is set lower than when the number of times does not exceed the specified number.

[0062] FIG. 7D is a schematic diagram illustrating how to give the reward rt according to (4) above (the reward rt is given so that the greater the deviation from the ideal rate of decrease in the supplier's available supply amount in a predetermined period, the lower the reward rt is).

[0063] When the reward rt is given using the method (4) described above, for example, an ideal rate of decrease until the end of the contract period is set, and the reward rt is given according to a value that quantifies the degree of deviation from that value. In the figure, (a) is an example when the rate of decrease is lower than the ideal rate of decrease, (b) is an example when the rate of decrease is higher than the ideal rate of decrease, and (c) is an example when the remaining coal amount becomes 0 during the contract period.

[0064] FIG. 8 shows examples of the evaluation indexes (1) to (4) for each state St, the action at, the action value Q(s, a), and the reward rt when the model learning process S800 is executed.

[0065] <Creating a Gantt chart> 9 is a diagram illustrating an example of a process or workflow performed using the planning support device 100, from the generation of the model 113 to the creation of a shipping plan (Gantt chart (operation control table)) using the model 113. The following description will be given with reference to this diagram.

[0066] First, the model learning unit 130 of the planning support device 100 learns the model 113 using the model input information 111 (S911 to S912).

[0067] Next, the shipping plan generating unit 135 inputs the initial values ​​of the model input information 111 into the trained model 113 to generate a shipping plan (S913). If the model 113 needs to be modified, the shipping plan generating unit 135 modifies the model 113 (S914).

[0068] Next, the shipping plan correction unit 140 corrects the shipping plan generated by the shipping plan generation unit 135 (S915). If the model 113 needs to be corrected, the shipping plan correction unit 140 corrects the model 113 (S916).

[0069] <Technical effects, etc.> As described above, the planning support device 100 generates a model for generating a shipping plan for a transport ship by performing deep reinforcement learning based on the following: coal yard information, including the amount of coal depleted at each coal yard; coal contract information, including the available supply volume of each of one or more coal suppliers and the number of days required for a transport ship to travel to the coal yard; ship information, including the load capacity and available unloading date of each transport ship; and ship operation information, including information indicating the daily operation status of each transport ship. The model then generates shipping plans for each transport ship using the generated model. This enables the generation of highly accurate shipping plans. Furthermore, because highly accurate shipping plans can be automatically generated without requiring skilled personnel, it is possible to reduce costs (human costs, time costs, etc.) and simplify technology transfer. Furthermore, it is possible to prevent performance degradation, for example, when on-site personnel are replaced. In this way, the planning support device 100 of this embodiment can support the planning of a shipping plan.

[0070] Furthermore, in the deep reinforcement learning of the model 113, the planning support device 100 gives a reward r that is higher when the remaining coal amount in the coal yard is within a predetermined appropriate range than when it is outside the range, so that it is possible to plan a shipping plan so that the remaining coal amount in the coal yard is maintained at an appropriate value.

[0071] Furthermore, in the deep reinforcement learning of the model 113, the planning support device 100 assigns a higher reward rt the smaller the difference between the amount of coal lost in the coal yard and the amount of coal supplied from the supplier, and therefore can plan a shipping plan so that the remaining amount of coal in the coal yard is maintained at an appropriate value.

[0072] Furthermore, in the deep reinforcement learning of the model 113, the planning support device 100 assigns a lower reward rt the more times coal whose quality is below a preset standard is continuously unloaded, so that the quality of the coal in the coal yard can be maintained at a required level and the generator output can be maintained stable at all times.

[0073] Furthermore, in the deep reinforcement learning of the model 113, the planning support device 100 assigns a lower reward rt the greater the deviation from the ideal rate of decrease in the supplier's supplyable amount in a specified period, so that it is possible to plan a shipping plan so that the remaining amount of coal in the coal yard is maintained at an appropriate value.

[0074] <Example of information processing device> FIG. 10 shows an example of the hardware configuration of an information processing device used to realize the components of the planning support system 1 (the planning support device 100, the coal storage monitoring device 200, and the ship allocation management device 300). The illustrated information processing device 10 includes a processor 11, a main memory device 12, an auxiliary memory device 13, an input device 14, an output device 15, and a communication device 16. Specific examples of the information processing device 10 include personal computers, office computers, various server devices, and general-purpose computers. All or part of the information processing device 10 may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. Each component of the planning support system 1 may be realized using, for example, multiple information processing devices 10 connected to each other so as to be able to communicate with each other.

[0075] In the same figure, the processor 11 is configured using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), an AI (Artificial Intelligence) chip, etc.

[0076] The main memory device 12 is a device that stores programs and data, and is, for example, a read-only memory (ROM), a random access memory (RAM), or a non-volatile memory (NVRAM (Non-Volatile RAM)).

[0077] The auxiliary storage device 13 is, for example, an SSD (Solid State Drive), a hard disk drive, an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage system, a reading / writing device for a recording medium such as an IC card, an SD card, or an optical recording medium, a storage area of ​​a cloud server, etc. Programs and data can be read into the auxiliary storage device 13 via a recording medium reading device or a communication device 16. The programs and data stored (memorized) in the auxiliary storage device 13 are read into the main storage device 12 as needed.

[0078] The input device 14 is an interface that accepts input from the outside, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a pen-input tablet, a voice input device, or the like.

[0079] The output device 15 is an interface that outputs various information such as the progress of processing and the results of processing. The output device 15 is, for example, a display device (LCD (Liquid Crystal Display), graphic card, etc.) that visualizes the various information, a device that converts the various information into audio (audio output device (speaker, etc.)), or a device that converts the various information into text (printer, etc.). Note that, for example, the information processing device 10 may be configured to input and output information to and from other devices via the communication device 16.

[0080] The input device 14 and the output device 15 constitute a user interface that receives information from the user and presents information to the user.

[0081] The communication device 16 is a device that realizes communication (wired communication or wireless communication) with other devices via a communication infrastructure such as a communication network 5, and is configured using, for example, a NIC (Network Interface Card), a wireless communication module, a USB module, etc.

[0082] The information processing device 10 may be equipped with, for example, an operating system, a file system, a DBMS (DataBase Management System) (relational database, NoSQL, etc.), a KVS (Key-Value Store), etc.

[0083] The functions of the components of the planning support system 1 (planning support device 100, coal storage monitoring device 200, ship allocation management device 300) are realized by the processor 11 of the information processing device 10 reading and executing a program stored in the main memory device 12, or by the functions of the hardware (FPGA, ASIC, AI chip, etc.) that constitutes the information processing device 10. The information processing device 10 stores various types of information (data), for example, as tables in a database or files managed by a file system.

[0084] <Summary> Although the embodiments of the present invention have been described in detail above, the above description is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention naturally includes equivalents thereof. For example, the above embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add, delete, or replace some of the configurations of the above embodiments with other configurations.

[0085] For example, the above describes a system that supports the planning of transport ship allocation plans to ensure that the amount of coal remaining in a coal stockpile is sufficient to ensure stable operation of a coal-fired power plant. However, the system can also be expanded to support the planning of transport ship allocation plans to ensure that the amount of fuel required to ensure stable operation of a power plant that uses other types of fuel, such as oil or natural gas, is stably maintained in storage facilities. [Explanation of symbols]

[0086] 1. Planning support system 5. Communication Network 10. Information processing equipment 100 Planning support device 110 Storage section 1111 Coal storage yard information 1112 Coal Contract Information 1113 Transport Ship Information 1114 Transport ship operation information 112 Reward Information 113 model 114 Ship Allocation Plan 125 Information Acquisition Management Department 130 Model Learning Department 135 Ship Allocation Plan Generation Unit 140 Vessel Allocation Plan Revision Department 200 Coal yard monitoring equipment 300 Ship allocation management device S600 Model Learning Process

Claims

1. A system for supporting the creation of a coal transport ship allocation plan, The information processing device includes a processor and a storage device, The environment is one or more coal yards into which coal is unloaded; one or more transport ship dispatchers as agents; a state based on coal stockpile information including the amount of coal depleted in each of the coal stockpile, coal contract information including the amount of coal that can be supplied from each of one or more coal suppliers and the number of days required for a transport ship to the coal stockpile to sail, ship information including the loading capacity and possible unloading date of each of the transport ships, and ship operation information including information showing the daily operation status of each of the transport ships, is defined as state St; The operational status of each transport ship is defined as action at, A model for generating a shipping plan for the transport ship is generated by performing deep reinforcement learning to obtain an action value Q(S, a) by assigning a reward rt according to the state S t+1 after the action at is performed, An initial value of the state St is input into the model to generate a shipping plan for each of the transport ships. Ship allocation planning support system.

2. 2. The vessel allocation planning support system according to claim 1, In the deep reinforcement learning, the action at is selected from at least one of the transport ship's demurrage, unloading, voyage, and two-port unloading, Ship allocation planning support system.

3. 2. The vessel allocation planning support system according to claim 1, In the deep reinforcement learning, the reward rt is given so that it is higher when the remaining amount of coal in the coal yard is within a predetermined appropriate range than when it is outside the range. Ship allocation planning support system.

4. 2. The vessel allocation planning support system according to claim 1, In the deep reinforcement learning, the smaller the difference between the amount of coal reduction in the coal yard and the amount of coal supplied from the supplier, the higher the reward rt is given. Ship allocation planning support system.

5. 2. The vessel allocation planning support system according to claim 1, In the deep reinforcement learning, the reward rt is given so that the reward rt becomes lower as the number of consecutive times that coal whose quality is equal to or lower than a preset reference value is unloaded increases. Ship allocation planning support system.

6. 2. The vessel allocation planning support system according to claim 1, In the deep reinforcement learning, the reward rt is given so as to become lower as the deviation from an ideal decrease rate of the supplyable amount in a predetermined period of the supplier increases. Ship allocation planning support system.

7. A method for supporting the creation of a coal transport ship allocation plan, comprising: An information processing device having a processor and a storage device, The environment is one or more coal yards into which coal is unloaded; one or more transport ship dispatchers as agents; a state based on coal stockpile information including the amount of coal depleted in each of the coal stockpile, coal contract information including the amount of coal that can be supplied from each of one or more coal suppliers and the number of days required for a transport ship to the coal stockpile to sail, ship information including the loading capacity and possible unloading date of each of the transport ships, and ship operation information including information showing the daily operation status of each of the transport ships, is defined as state St; The operational status of each transport ship is defined as action at, A step of generating a model for generating a shipping plan for the transport ships by performing deep reinforcement learning to obtain an action value Q(S, a) by assigning a reward rt according to the state S t+1 after performing the action at; generating a shipping plan for each of the transport ships by inputting an initial value of the state St into the model; A method for supporting ship allocation planning.

8. The vessel allocation planning support method according to claim 7, In the deep reinforcement learning, the action at is selected from at least one of the transport ship's demurrage, unloading, voyage, and two-port unloading, Ship allocation planning support method.

9. The vessel allocation planning support method according to claim 7, In the deep reinforcement learning, the reward r is given so that it is higher when the remaining amount of coal in the coal yard is within a predetermined appropriate range than when it is outside the range. Ship allocation planning support method.

10. The vessel allocation planning support method according to claim 7, In the deep reinforcement learning, the smaller the difference between the amount of coal reduction in the coal yard and the amount of coal supplied from the supplier, the higher the reward rt is given. Ship allocation planning support method.

11. The vessel allocation planning support method according to claim 7, In the deep reinforcement learning, the reward rt is given so that the reward rt becomes lower as the number of consecutive times that coal whose quality is equal to or lower than a preset reference value is unloaded increases. Ship allocation planning support method.

12. The vessel allocation planning support method according to claim 7, In the deep reinforcement learning, the reward rt is given so as to become lower as the deviation from an ideal decrease rate of the supplyable amount in a predetermined period of the supplier increases. Ship allocation planning support method.

Images