Material Scheduling Method, Model Training Method and Device

Abstract

The present disclosure provides a material scheduling method, a model training method and an apparatus, which relate to the field of artificial intelligence technology, and particularly to the fields of simulation control and reinforcement learning technology. The specific implementation solution is as follows: obtaining the state information corresponding to the target port; determining a set of material scheduling parameters matching the state information based on the state information and the trained port material scheduling model; and performing port material scheduling operations based on each material scheduling parameter in the set of material scheduling parameters. This implementation manner can improve the efficiency and accuracy of port material scheduling.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and further relates to the fields of simulation control and reinforcement learning technology. Specifically, it relates to a material scheduling method, a model training method and a device. Background Art

[0002] Currently, in the port environment, when performing material scheduling, the following process needs to be executed: train arrival at the port - train unloading - temporary storage of materials - ship loading.

[0003] In the above material scheduling process, various parameters of port operations need to be adjusted. However, the current method of adjusting various parameters of port operations relies on manual adjustment, resulting in low efficiency and poor accuracy of material scheduling. Summary of the Invention

[0004] The present disclosure provides a material scheduling method, a model training method and a device.

[0005] According to one aspect of the present disclosure, a material scheduling method is provided, including: obtaining state information corresponding to a target port; determining a set of material scheduling parameters matching the state information based on the state information and a trained port material scheduling model; and performing a port material scheduling operation based on each material scheduling parameter in the set of material scheduling parameters.

[0006] According to another aspect of the present disclosure, a model training method is provided, including: obtaining sample state information; performing the following model training steps on the sample state information: determining a set of sample material scheduling parameters matching the sample state information based on the sample state information and a model to be trained; determining a reward value based on the sample state information, the set of sample material scheduling parameters and a preset reward function; and in response to determining that the reward value meets a preset convergence condition, determining the model to be trained as a trained port material scheduling model.

[0007] According to another aspect of the present disclosure, a device for material scheduling is provided, including: a state acquisition unit configured to obtain state information corresponding to a target port; a parameter determination unit configured to determine a set of material scheduling parameters matching the state information based on the state information and a trained port material scheduling model; and a material scheduling operation unit configured to perform a port material scheduling operation based on each material scheduling parameter in the set of material scheduling parameters.

[0008] According to another aspect of the present disclosure, there is provided a model training device, including: a sample status acquisition unit configured to acquire sample status information; a model training unit configured to perform the following model training steps on the sample status information: determining a set of sample material scheduling parameters matching the sample status information based on the sample status information and the model to be trained; determining a reward value based on the sample status information, the set of sample material scheduling parameters, and a preset reward function; and in response to determining that the reward value satisfies a preset convergence condition, determining the model to be trained as a trained port material scheduling model.

[0009] According to another aspect of the present disclosure, there is provided an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, causing the one or more processors to implement any of the above material scheduling methods or model training methods.

[0010] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute any of the above material scheduling methods or model training methods.

[0011] According to another aspect of the present disclosure, there is provided a computer program product including a computer program, which when executed by a processor implements any of the above material scheduling methods or model training methods.

[0012] According to the technology of the present disclosure, there is provided a material scheduling method that can improve the efficiency and accuracy of port material scheduling.

[0013] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0015] Figure 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

[0016] Figure 2 is a flowchart of an embodiment of the material scheduling method according to the present disclosure;

[0017] Figure 3 is a schematic diagram of an application scenario of the material scheduling method according to the present disclosure;

[0018] Figure 4It is a flowchart of an embodiment of the model training method according to the present disclosure;

[0019] Figure 5 It is a flowchart of another embodiment of the model training method according to the present disclosure;

[0020] Figure 6 It is a schematic structural diagram of an embodiment of the material scheduling device according to the present disclosure;

[0021] Figure 7 It is a schematic structural diagram of an embodiment of the model training device according to the present disclosure;

[0022] Figure 8 It is a block diagram of an electronic device for implementing the material scheduling method or the model training method of the embodiments of the present disclosure. Detailed implementation manners

[0023] The following makes an illustration of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted below.

[0024] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0025] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0026] The terminal devices 101, 102, and 103 interact with the server 105 via the network 104 to receive or send messages, etc. Among them, the terminal devices 101, 102, and 103 can be used in the material scheduling scenario, determine the corresponding set of material scheduling parameters based on the status information of the target port, and use each material scheduling parameter in the set of material scheduling parameters to control the corresponding port material scheduling device to perform the corresponding port material scheduling operation. In practical applications, the terminal devices 101, 102, and 103 can send the status information corresponding to the target port to the server 105 via the network 104, so that the server 105 determines the set of material scheduling parameters based on the status information and the trained port material scheduling model, and returns the set of material scheduling parameters to the terminal devices 101, 102, and 103, so that the terminal devices 101, 102, and 103 perform the port material scheduling operation based on each material scheduling parameter in the set of material scheduling parameters. Or, in the model training stage, the terminal devices 101, 102, and 103 can also send the sample status information to the server 105 via the network 104, so that the server 105 performs model training based on the sample status information.

[0027] The terminal devices 101, 102, and 103 can be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to mobile phones, computers, tablets, etc. When the terminal devices 101, 102, and 103 are software, they can be installed in the above-listed electronic devices. It can be implemented as multiple software or software modules (for example, used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0028] The server 105 can be a server that provides various services. For example, the server 105 can obtain the status information sent by the terminal devices 101, 102, and 103, determine the set of material scheduling parameters that match the status information based on the status information and the trained port material scheduling model, and return the set of material scheduling parameters to the terminal devices 101, 102, and 103 via the network 104. Or, in the model training stage, the server 105 can also receive the sample status information sent by the terminal devices 101, 102, and 103, and use the sample status information to train the model to be trained to obtain the trained port material scheduling model.

[0029] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0030] It should be noted that the material scheduling method or model training method provided by the embodiments of the present disclosure can be executed by the terminal devices 101, 102, and 103, or can be executed by the server 105. The device for material scheduling or the model training device can be set in the terminal devices 101, 102, and 103, or can be set in the server 105.

[0031] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0032] Continuing to refer to Figure 2 , a process 200 of the material scheduling method according to an embodiment of the present disclosure is shown. The material scheduling method of this embodiment includes the following steps:

[0033] Step 201, obtaining the status information corresponding to the target port.

[0034] In this embodiment, the execution subject (such as Figure 1 the terminal devices 101, 102, and 103 or the server 105 in

[0035] In some optional implementation manners of this embodiment, the status information includes at least one of the following: stacking status information, car dumper operation status information, belt operation status information, reclaimer operation status information, ship loader operation status information.

[0036] In this implementation manner, the stacking state information may include the quantity of stacked materials, the weight of stacked materials, the stacking number of materials, the material category information corresponding to the stack, etc.; the operation state information of the car dumper may include the car dumper number, the start time of the car dumper operation, whether the car dumper has started operating, etc.; the operation state information of the belt may include the belt number, the occupation time of the belt, whether the belt is idle, etc.; the operation state of the reclaimer may include the reclaimer number, the start time of the reclaimer operation, whether the reclaimer has started operating, etc.; the operation state information of the ship loader may include the ship loader number, the start time of the ship loader operation, whether the ship loader has started operating, etc.

[0037] Step 202: Based on the state information and the trained port material scheduling model, determine a set of material scheduling parameters that match the state information.

[0038] In this embodiment, the execution entity may input the above state information into the trained port material scheduling model, so that the trained port material scheduling model generates a set of material scheduling parameters corresponding to the state information, and execute the port material scheduling operation according to each material scheduling parameter in the set of material scheduling parameters. Among them, the trained port material scheduling model is a pre-trained model for realizing the simultaneous control of multiple parameters, and specifically, a DQN reinforcement learning model (Deep Q Network, a model that combines a neural network and Q-learning), an actor-critic model (a model implemented based on reinforcement learning), etc. may be adopted. This embodiment does not make any limitations in this regard.

[0039] In some alternative implementation manners of this embodiment, each material scheduling parameter in the set of material scheduling parameters includes at least one of the following: freight train parameters, train-carrying material category parameters, belt parameters, car dumper parameters, stacking parameters, unloader parameters, discharge trolley parameters, reclaimer parameters, activated feeder parameters, ship loader parameters, inbound ship order parameters.

[0040] In this implementation manner, the freight train parameters may include parameters such as the train number of the train and the train model; the material category parameters carried by the train may include the material category parameters; the belt parameters may include parameters such as the belt number selected when performing port material scheduling operations and the start time of belt operations; the dumper parameters may include the dumper number selected when performing port material scheduling operations and the start time of dumper operations; the stacking parameters may include the stacking number selected when performing port material scheduling operations; the unloader parameters may include the unloader number selected when performing port material scheduling operations; the activated feeder parameters may include the activated feeder number selected when performing port material scheduling operations; the ship loader parameters may include the ship loader number selected when performing port material scheduling operations, and the ship entry list parameters may include the ship loading parameters selected when performing port material scheduling operations.

[0041] Step 203, based on each material scheduling parameter in the material scheduling parameter set, perform port material scheduling operations.

[0042] In this embodiment, after determining the material scheduling parameter set, the execution entity may also execute each process of the port material scheduling operation according to each material scheduling parameter in the material scheduling parameter set.

[0043] In some optional implementation manners of this embodiment, performing port material scheduling operations based on each material scheduling parameter in the material scheduling parameter set may include: based on the material transportation parameters in the material scheduling parameter set, select a specified device for transporting materials; based on the material storage parameters in the material scheduling parameter set, control the specified device to transport the materials to the specified storage location; based on the material ship loading parameters in the material scheduling parameter set, select a specified device for pulling goods and loading them onto the ship, and use the specified device to load the materials at the specified storage location into the corresponding ship.

[0044] Continue to refer to Figure 3 , which shows a schematic diagram of an application scenario of the material scheduling method according to the present disclosure. In Figure 3In the application scenario, the execution entity can obtain the target port 301 that needs to perform port material scheduling, and obtain the status information 302 of the target port 301, such as the operation status of the car dumper, the operation status of the belt, the stacking status, the operation status of the reclaimer, the operation status of the ship loader, etc. Among them, these status information 302 can be manually input, or the preset sensing device can obtain the sensing data of the target port 301 and analyze it to obtain. This embodiment does not limit the specific acquisition method of the status information 302. After that, the execution entity can input the status information 302 into the port material scheduling model 303 to obtain the material scheduling parameters 304 output by the model. The material scheduling parameters 304 can include freight train parameters, train-borne material category parameters, belt parameters, car dumper parameters, stacking parameters, unloader parameters, etc. The execution entity can control the material scheduling equipment to perform port material scheduling operations 305 according to the material scheduling parameters 304 by establishing a communication connection with the material scheduling equipment in the target port 301 in advance.

[0045] The material scheduling method provided by the above embodiment of the present disclosure can consider the entire process of port material scheduling operations as a whole, use a pre-trained port material scheduling model to generate corresponding material scheduling parameters, and schedule the equipment in the port according to the material scheduling parameters to perform port material scheduling operations, thereby improving the efficiency and accuracy of port material scheduling.

[0046] Continue to refer to Figure 4 , which shows a flow 400 of an embodiment of the model training method according to the present disclosure. As Figure 4 shown, the model training method of this embodiment may include the following steps:

[0047] Step 401, obtain sample status information.

[0048] In this embodiment, the sample status information may be information on the working status of equipment for transporting materials and the storage status of equipment for storing materials in a simulated real port. For a detailed description of the sample status information, please refer to the detailed description of the status information corresponding to the target port for details, and will not be repeated here.

[0049] Among them, since model training needs to be iterated multiple times, the sample status information obtained here is the initial status information. After one round of model training, the sample status information will be updated and the next round of model training will be carried out again until the model training is completed.

[0050] Step 402, perform the following model training steps on the sample status information: Based on the sample status information and the model to be trained, determine a set of sample material scheduling parameters that match the sample status information; Based on the sample status information, the set of sample material scheduling parameters, and a preset reward function, determine a reward value; In response to determining that the reward value meets the preset convergence condition, determine the model to be trained as the trained port material scheduling model.

[0051] In this embodiment, the model to be trained can be a DQN reinforcement learning model. The execution entity can input the sample status information into the model to be trained and obtain a set of sample material scheduling parameters output by the model to be trained. For a detailed description of the set of sample material scheduling parameters, please refer to the detailed description of the set of material scheduling parameters and will not be elaborated here. After that, the execution entity can determine the reward value based on the sample status information, the set of sample material scheduling parameters, and the preset reward function. If the reward value does not meet the preset convergence condition, update the sample status information and repeat the model training steps until the reward value meets the preset convergence condition to obtain the trained port material scheduling model. Among them, the model to be trained can also be other reinforcement learning models, and this embodiment does not limit this.

[0052] The model training method provided in the above embodiment of the present disclosure can also train the model to be trained through the sample status information and the preset reward function to obtain a port material scheduling model, so as to realize the comprehensive scheduling of port material scheduling and improve the port material scheduling effect.

[0053] Continue to refer to Figure 5 which shows a flow 500 of another embodiment of the model training method according to the present disclosure. As Figure 5 shown, the model training method of this embodiment can include the following steps:

[0054] Step 501, obtain sample status information.

[0055] In this embodiment, the sample status information includes at least one of the following: stacker sample status information, dumper sample operation status information, belt sample operation status information, reclaimer sample operation status information, shiploader sample operation status information.

[0056] Among them, for the detailed description of the sample status information, please refer to the detailed description of the status information of the target port; for the detailed description of the stack sample status information, please refer to the detailed description of the stack status information; for the detailed description of the dumper sample operation status information, please refer to the detailed description of the dumper operation status information; for the detailed description of the belt sample operation status information, please refer to the detailed description of the belt operation status information; for the detailed description of the reclaimer sample operation status information, please refer to the detailed description of the reclaimer operation status information; for the detailed description of the ship loader sample operation status information, please refer to the detailed description of the ship loader operation status information.

[0057] Step 502, perform the following model training steps on the sample status information: Based on the sample status information and the model to be trained, determine the set of sample material scheduling parameters that match the sample status information; Based on the sample status information, the set of sample material scheduling parameters, and a preset reward function, determine the reward value; In response to determining that the reward value meets the preset convergence condition, determine the model to be trained as the trained port material scheduling model.

[0058] In this embodiment, each sample material scheduling parameter in the set of sample material scheduling parameters includes at least one of the following: freight train sample parameter, train-carried material category sample parameter, belt sample parameter, dumper sample parameter, stack sample parameter, unloader sample parameter, unloader trolley sample parameter, reclaimer sample parameter, activated feeder sample parameter, ship loader sample parameter, incoming ship list sample parameter. Among them, for the detailed description of each sample material scheduling parameter in the set of sample material scheduling parameters, please refer to the detailed description of each material scheduling parameter in the set of material scheduling parameters, which will not be elaborated here.

[0059] Step 503, in response to determining that the reward value does not meet the preset convergence condition, update the sample status information based on the simulation environment, and perform the model training steps on the updated sample status information until the trained port material scheduling model is obtained.

[0060] In this embodiment, if the above reward value does not meet the preset convergence condition, the sample status information is updated using the simulation environment, and the above model training steps are repeatedly performed on the updated sample status information until the trained port material scheduling model is obtained. Among them, the simulation environment is used to simulate the real scenario of port material scheduling.

[0061] In some alternative implementation manners of this embodiment, updating the sample status information based on the simulation environment includes: Based on the set of sample material scheduling parameters, controlling the simulation environment to simulate the port material scheduling operation, obtaining the simulation environment after simulating the port material scheduling; Based on the simulation environment after simulating the port material scheduling, updating the sample status information.

[0062] In this implementation, when the execution entity finishes one round of training and the reward value does not meet the preset convergence condition, it can use the set of sample material scheduling parameters obtained in this round of training to control each device in the simulation environment to perform port material scheduling operations according to the sample material scheduling parameters, and update the above-mentioned sample state information based on the simulation environment after simulating port material scheduling. By using this method of simulating and updating the sample state information in the simulation environment, the update of the sample state information can be made more in line with the actual port material scheduling operation, thereby improving the training effect of the model.

[0063] Moreover, the port material scheduling operation can include unloading operations and cargo pulling operations. For unloading operations, the stacker sample state information can be updated based on the following formula:

[0064]

[0065] where W i represents the updated number of stacker samples, and W i-1 represents the number of stacker samples before the update. represents the quantity of materials unloaded by n unloaders.

[0066] In addition, the dumper sample operation state information is updated based on the following formula:

[0067] st i = st i-1 + t_wt

[0068] where st i represents the updated dumper start operation time, st i-1 represents the dumper start operation time before the update, and t_wt represents the operation duration of the dumper before the update.

[0069] In addition, the start operation time of the belt sample can be equal to the updated dumper start operation time.

[0070] For cargo pulling operations, the stacker sample state information can be updated based on the following formula:

[0071]

[0072] where W i represents the updated number of stacker samples, and W i-1 represents the number of stacker samples before the update. represents the quantity of materials pulled away by n cargo pullers.

[0073] In addition, the sample operation state information of the reclaimer or the activated feeder is updated based on the following formula:

[0074] st i = st i-1 + b_wt

[0075] wherein, st i represents the start operation time of the updated reclaiming machine or the activated feeder, and st i-1 represents the start operation time of the reclaiming machine or the activated feeder before the update, and b_wt represents the operation duration of the reclaiming machine or the activated feeder before the update.

[0076] Also, the belt occupancy time can be equal to the start operation time of the updated car dumper.

[0077] For example, when controlling the simulation environment to simulate the car unloading scenario according to the sample material scheduling parameter set, parameters such as the number of trains, the material category corresponding to each train, the arrival time of each train, the train type of each train, and the number of trains in the car entry list can be determined according to the sample material scheduling parameter set, and for the unloading operation of each train, corresponding resources such as car dumpers, belts, unloading machines, and stackers are selected. Each belt and each car dumper have corresponding start operation times, as well as the operation duration corresponding to the train type (which can be determined through a preset correspondence table between vehicle attributes and operation durations). Based on the start operation time and the operation duration, the operation end time can be obtained. For example, if the operation end time of a certain belt and car dumper is 6:38, then the next start operation time of this belt and car dumper is 6:39.

[0078] In some other alternative implementation manners of this embodiment, based on the sample material scheduling parameter set, the simulation environment is controlled to simulate the port material scheduling operation, and the simulation environment after simulating the port material scheduling is obtained, including: configuring the operation parameters of the target devices in the simulation environment based on the sample material scheduling parameter set and the preset constraint conditions; controlling the target devices to operate according to the operation parameters to obtain the simulation environment after simulating the port material scheduling.

[0079] In this implementation manner, the target devices can be each device that simulates the port material scheduling operation in the simulation environment, and can include but are not limited to trains, belt conveyors, car dumpers, unloading machines, unloading trolleys, ships for pulling goods, reclaiming machines, activated feeders, etc., and this embodiment does not make any limitations in this regard. The preset constraint conditions are used to limit the selection of each device in the simulation environment to achieve the reasonable scheduling of each device in the simulation environment. The execution entity can select the target devices required for this round of simulating the port material scheduling operation in the simulation environment according to the preset constraint conditions and the sample material scheduling parameter set, and configure the operation parameters of the target devices so that the target devices operate according to the corresponding operation parameters to obtain the simulation environment after simulating the port material scheduling.

[0080] In some other alternative implementation manners of this embodiment, the preset constraint conditions at least include: the target device is an available device; and / or, the operation time of the target device meets the preset time conditions; and / or, the device type of the target device matches the sample material scheduling parameters in the sample material scheduling parameter set.

[0081] In this implementation manner, the available device can be a device that is in an idle state and can be selected. By restricting the target device to be an available device, the rationality of device determination can be improved. Moreover, the operation time of the target device can include the start operation time, arrival time, car coupling time, etc. of the target device, and this embodiment does not limit this. The preset time conditions can include that the start operation time of the target device is greater than or equal to the sum of the arrival time and the car coupling time of the target device, or the preset time conditions can also include that the sum of the start operation time and the car coupling time of the target device is within a preset time range. By using this kind of constraint on the time range, the real port material scheduling operation situation in different time periods can be simulated. In addition, the device type of the target device can include but is not limited to the equipment height limit type, equipment attribute type, material category of equipment material scheduling, etc., and this embodiment does not limit this.

[0082] Among them, in response to determining that the target device is a ten - thousand - ton train, the constraint conditions for the target device can further include: if the tippers corresponding to the single trains of the same ten - thousand - ton train are the same, there is no car coupling time; and / or, the tippers corresponding to the ten - thousand - ton train are of a type suitable for ten - thousand - ton trains.

[0083] In some other alternative implementation manners of this embodiment, it further includes: determining a preset reward function based on the total amount of train material scheduling, the total amount of ship unloading, and the belt operation interval within a preset time period.

[0084] In this implementation manner, the preset reward function can be determined based on the total amount of transported materials and the resource waiting duration. Specifically, the expected goal can be set as the maximum total amount of transported materials and the minimum resource waiting duration. Among them, the preset reward function can be as follows:

[0085]

[0086] Among them, reward represents the expected function, ST i represents the start operation time of the i - th train, T represents the preset time, represents the start operation times of all trains within a certain time, tw i represents the weight of the material scheduling of the i - th train, y i represents whether the i - th train is selected for material scheduling. If it is selected, it is 1; if it is not selected, it is 0, bw j represents the material category of the material scheduling of the j - th ship, kj It indicates whether the j-th train is selected for material scheduling. If it is selected, the value is 1; if not, the value is 0. n represents the total number of trains, and m represents the total number of ships. For example, when conducting model training, if simulating the port material scheduling situation within the time range from 7:00 to 18:30, and taking 1 minute as the interval time, T can be 660 at this time, representing the total number of 1-minute intervals from 7:00 to 18:30. If the total number of trains is constant within this time range, the longer the interval time between trains, the higher the reward value.

[0087] The model training method provided by the above embodiments of the present disclosure can also simulate port material scheduling operations using a simulation environment and a sample set of material scheduling parameters, update the sample state information based on the simulation environment, and improve the authenticity of the update of the sample state information. Moreover, by setting constraint conditions to configure the operation parameters of the target devices in the simulation environment, the rationality of the configuration of the operation parameters can be improved, and further the accuracy of the update of the sample state information can be improved. Additionally, based on the total amount of train material scheduling, the total amount of ship unloading, and the belt operation interval within a preset time period, the model training objectives can be established from two aspects: the total amount of material scheduling and the waiting duration, improving the strategy generation effect of the trained model.

[0088] Further referring to Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a material scheduling device. This device embodiment corresponds to the Figure 2 method embodiment shown, and this device can be specifically applied to electronic devices such as terminal devices and servers.

[0089] As Figure 6 shown, the material scheduling device 600 of this embodiment includes: a state acquisition unit 601, a parameter determination unit 602, and a material scheduling operation unit 603.

[0090] The state acquisition unit 601 is configured to acquire the state information corresponding to the target port.

[0091] The parameter determination unit 602 is configured to determine a set of material scheduling parameters that match the state information based on the state information and the trained port material scheduling model.

[0092] The material scheduling operation unit 603 is configured to perform port material scheduling operations based on each material scheduling parameter in the set of material scheduling parameters.

[0093] In some optional implementation manners of this embodiment, the state information includes at least one of the following: stacking state information, dumper operation state information, belt operation state information, reclaimer operation state information, and ship loader operation state information.

[0094] In some alternative implementation manners of this embodiment, each material scheduling parameter in the material scheduling parameter set includes at least one of the following: freight train parameter, train-carried material category parameter, belt parameter, dumper parameter, stacking parameter, unloader parameter, tripper car parameter, reclaimer parameter, activated feeder parameter, ship loader parameter, and incoming ship list parameter.

[0095] It should be understood that the units 601 to 603 described in the device 600 for material scheduling respectively correspond to the respective steps in the method described in the reference Figure 2 Therefore, the operations and features described above for the material scheduling method also apply to the device 600 and the units included therein, and will not be elaborated herein.

[0096] Further referring to Figure 7 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a model training device. This device embodiment corresponds to the method embodiment shown in Figure 4 , and this device can be specifically applied to electronic devices such as terminal devices and servers.

[0097] As shown in Figure 7 , the model training device 700 in this embodiment includes: a sample status acquisition unit 701 and a model training unit 702.

[0098] The status acquisition unit 701 is configured to acquire sample status information.

[0099] The model training unit 702 is configured to perform the following model training steps on the sample status information: determine a set of sample material scheduling parameters that match the sample status information based on the sample status information and the model to be trained; determine a reward value based on the sample status information, the set of sample material scheduling parameters, and a preset reward function; and in response to determining that the reward value meets the preset convergence condition, determine the model to be trained as the trained port material scheduling model.

[0100] In some alternative implementation manners of this embodiment, the model training unit 702 is further configured to: in response to determining that the reward value does not meet the preset convergence condition, update the sample status information based on the simulation environment, and perform the model training steps on the updated sample status information until the trained port material scheduling model is obtained.

[0101] In some alternative implementation manners of this embodiment, the model training unit 702 is further configured to: based on the set of sample material scheduling parameters, control the simulation environment to simulate port material scheduling operations, obtain the simulation environment after simulating port material scheduling; and update the sample status information based on the simulation environment after simulating port material scheduling.

[0102] In some alternative implementation manners of this embodiment, the model training unit 702 is further configured to: configure the operation parameters of the target device in the simulation environment based on the sample material scheduling parameter set and the preset constraint conditions; control the target device to operate according to the operation parameters, and obtain the simulation environment after simulating the port material scheduling.

[0103] In some alternative implementation manners of this embodiment, the preset constraint conditions at least include: the target device is an available device; and / or, the operation time of the target device meets the preset time conditions; and / or, the device type of the target device matches the sample material scheduling parameters in the sample material scheduling parameter set.

[0104] In some alternative implementation manners of this embodiment, the sample status information at least includes one of the following: stacker sample status information, car dumper sample operation status information, belt sample operation status information, reclaimer sample operation status information, ship loader sample operation status information.

[0105] In some alternative implementation manners of this embodiment, each sample material scheduling parameter in the sample material scheduling parameter set at least includes one of the following: freight train sample parameter, train carried material category sample parameter, belt sample parameter, car dumper sample parameter, stacker sample parameter, unloader sample parameter, tripper sample parameter, reclaimer sample parameter, activated feeder sample parameter, ship loader sample parameter, ship arrival order sample parameter.

[0106] In some alternative implementation manners of this embodiment, the model training unit 702 is further configured to: determine the preset reward function based on the total amount of goods pulled by the train, the total amount of goods unloaded by the ship, and the belt operation interval within the preset time period.

[0107] It should be understood that units 701 to 703 recorded in the model training apparatus 700 respectively correspond to each step in the method described in the reference Figure 4 Therefore, the operations and features described above for the model training method are equally applicable to the apparatus 700 and the units included therein, and will not be repeated here.

[0108] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0109] Figure 8FIG. 0 shows a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0111] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as, for example, a keyboard, a mouse, etc.; an output unit 807, such as, for example, various types of displays, speakers, etc.; a storage unit 808, such as, for example, a magnetic disk, an optical disk, etc.; and a communication unit 809, such as, for example, a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0112] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the material scheduling method or the model training method. For example, in some embodiments, the material scheduling method or the model training method can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the material scheduling method or the model training method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the material scheduling method or the model training method by any other suitable means (e.g., by means of firmware).

[0113] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program code is executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0117] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0118] A computer system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0119] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0120] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A material scheduling method, comprising: Obtaining the status information corresponding to the target port; Determining a set of material scheduling parameters matching the status information based on the status information and the trained port material scheduling model; Performing port material scheduling operations based on each material scheduling parameter in the set of material scheduling parameters; Wherein, the port material scheduling model is obtained through the following training steps: determining a set of sample material scheduling parameters matching the sample status information based on the sample status information and the model to be trained; determining a reward value based on the sample status information, the set of sample material scheduling parameters, and a preset reward function; in response to determining that the reward value meets the preset convergence condition, determining the model to be trained as the trained port material scheduling model; in response to determining that the reward value does not meet the preset convergence condition, updating the sample status information based on the simulation environment, and performing the model training steps on the updated sample status information until the trained port material scheduling model is obtained; Wherein, the updating the sample status information based on the simulation environment includes: Controlling the simulation environment to simulate port material scheduling operations based on preset constraint conditions and the set of sample material scheduling parameters, to obtain a simulation environment after simulated port material scheduling; Updating the sample status information based on the simulation environment after simulated port material scheduling; and The preset reward function is determined based on the total amount of goods transported by trains, the total amount of goods unloaded by ships, and the belt operation interval within a preset time period.

2. The method according to claim 1, wherein The status information at least includes one of the following: stacking status information, car dumper operation status information, belt operation status information, reclaimer operation status information, ship loader operation status information.

3. The method according to claim 1, wherein, Each material scheduling parameter in the set of material scheduling parameters at least includes one of the following: freight train parameters, train-carrying material category parameters, belt parameters, car dumper parameters, stacking parameters, unloader parameters, unloader trolley parameters, reclaimer parameters, activated feeder parameters, ship loader parameters, ship arrival order parameters.

4. A model training method, comprising: Obtaining sample status information; Performing the following model training steps on the sample status information: determining a set of sample material scheduling parameters matching the sample status information based on the sample status information and the model to be trained; Determining a reward value based on the sample status information, the set of sample material scheduling parameters, and a preset reward function; In response to determining that the reward value meets the preset convergence condition, determining the model to be trained as the trained port material scheduling model; In response to determining that the reward value does not meet the preset convergence condition, updating the sample status information based on the simulation environment, and performing the model training steps on the updated sample status information until the trained port material scheduling model is obtained; Wherein, the updating the sample status information based on the simulation environment includes: Controlling the simulation environment to simulate port material scheduling operations based on preset constraint conditions and the set of sample material scheduling parameters, to obtain a simulation environment after simulated port material scheduling; Update the sample status information based on the simulation environment after simulating port material scheduling; and The method further includes: Determine the preset reward function based on the total amount of goods transported by trains, the total amount of goods unloaded by ships, and the belt operation interval within a preset time period.

5. The method according to claim 4, wherein The controlling the simulation environment to simulate port material scheduling operations based on the preset constraint conditions and the sample material scheduling parameter set to obtain the simulation environment after simulating port material scheduling includes: Configure the operation parameters of the target device in the simulation environment based on the sample material scheduling parameter set and the preset constraint conditions; Control the target device to operate according to the operation parameters to obtain the simulation environment after simulating port material scheduling.

6. The method according to claim 5, wherein, The preset constraint conditions at least include: The target device is an available device; and / or The operation time of the target device meets the preset time conditions; and / or The device type of the target device matches the sample material scheduling parameters in the sample material scheduling parameter set.

7. The method according to claim 4, wherein The sample status information at least includes one of the following: stacker sample status information, dumper sample operation status information, belt sample operation status information, reclaimer sample operation status information, ship loader sample operation status information.

8. The method according to claim 4, wherein Each sample material scheduling parameter in the sample material scheduling parameter set at least includes one of the following: freight train sample parameter, train-carried material category sample parameter, belt sample parameter, dumper sample parameter, stacker sample parameter, unloader sample parameter, tripper sample parameter, reclaimer sample parameter, activation feeder sample parameter, ship loader sample parameter, incoming ship list sample parameter.

9. A device for material scheduling, comprising: A status acquisition unit configured to acquire the status information corresponding to the target port; A parameter determination unit configured to determine a material scheduling parameter set matching the status information based on the status information and the trained port material scheduling model; A material scheduling operation unit configured to perform port material scheduling operations based on each material scheduling parameter in the material scheduling parameter set; Wherein, the port material scheduling model is obtained through the following training steps: determining a sample material scheduling parameter set matching the sample status information based on the sample status information and the model to be trained; determining a reward value based on the sample status information, the sample material scheduling parameter set, and the preset reward function; in response to determining that the reward value meets the preset convergence condition, determining the model to be trained as the trained port material scheduling model; in response to determining that the reward value does not meet the preset convergence condition, updating the sample status information based on the simulation environment and performing the model training steps on the updated sample status information until the trained port material scheduling model is obtained; Among them, updating the sample state information based on the simulation environment includes: controlling the simulation environment to simulate port material scheduling operations based on preset constraint conditions and the sample material scheduling parameter set, to obtain a simulation environment after simulating port material scheduling; updating the sample state information based on the simulation environment after simulating port material scheduling; and The preset reward function is determined based on the total amount of goods transported by trains, the total amount of goods unloaded by ships, and the belt operation interval within a preset time period.

10. The apparatus according to claim 9, wherein, The state information includes at least one of the following: stacking state information, car dumper operation state information, belt operation state information, reclaimer operation state information, ship loader operation state information.

11. The device according to claim 9, wherein, Each material scheduling parameter in the material scheduling parameter set includes at least one of the following: freight train parameter, train-borne material category parameter, belt parameter, car dumper parameter, stacking parameter, unloader parameter, unloader trolley parameter, reclaimer parameter, activated feeder parameter, ship loader parameter, incoming ship list parameter.

12. A model training device, comprising: A sample state acquisition unit configured to acquire sample state information; A model training unit configured to perform the following model training steps on the sample state information: determining a sample material scheduling parameter set that matches the sample state information based on the sample state information and the model to be trained; Determining a reward value based on the sample state information, the sample material scheduling parameter set, and a preset reward function; in response to determining that the reward value meets the preset convergence condition, determining the model to be trained as a trained port material scheduling model; In response to determining that the reward value does not meet the preset convergence condition, updating the sample state information based on the simulation environment, and performing the model training steps on the updated sample state information until the trained port material scheduling model is obtained; Among them, the model training unit is further configured to: Controlling the simulation environment to simulate port material scheduling operations based on preset constraint conditions and the sample material scheduling parameter set, to obtain a simulation environment after simulating port material scheduling; Updating the sample state information based on the simulation environment after simulating port material scheduling; and The device further includes: A reward function determination module configured to determine the preset reward function based on the total amount of goods transported by trains, the total amount of goods unloaded by ships, and the belt operation interval within a preset time period.

13. The apparatus according to claim 12, wherein, The model training unit is further configured to: Configuring the operation parameters of the target device in the simulation environment based on the sample material scheduling parameter set and preset constraint conditions; Controlling the target device to operate according to the operation parameters to obtain a simulation environment after simulating port material scheduling.

14. The apparatus according to claim 13, wherein, The preset constraint conditions include at least: The target device is an available device; and / or The operation time of the target device meets the preset time condition; and / or The device type of the target device matches the sample material scheduling parameter in the sample material scheduling parameter set.

15. The apparatus according to claim 12, wherein, The sample status information at least includes one of the following: stack sample status information, car dumper sample operation status information, belt sample operation status information, reclaimer sample operation status information, ship loader sample operation status information.

16. The device according to claim 12, wherein, Each sample material scheduling parameter in the sample material scheduling parameter set at least includes one of the following: freight train sample parameter, train carried material category sample parameter, belt sample parameter, car dumper sample parameter, stack sample parameter, unloader sample parameter, tripper car sample parameter, reclaimer sample parameter, activated feeder sample parameter, ship loader sample parameter, incoming ship list sample parameter.

17. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8.

Images