A method, device, equipment and storage medium for determining a scheduling plan

Abstract

The present disclosure provides a method, device, equipment and storage medium for determining a scheduling scheme, which relates to the field of machine learning technology, and in particular to the field of scheduling technology and reinforcement learning technology. The specific implementation scheme is: obtaining status information, wherein the status information includes: description information of the operating belt included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port; based on the status information, determining a target scheduling scheme including the correspondence between the transport vehicle, the operating belt and the time. When the scheduling scheme is determined by applying the embodiment of the present disclosure, the efficiency of determining the scheduling scheme is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning, in particular to the field of scheduling technology and reinforcement learning technology, and in particular to a method, device, equipment and storage medium for determining a scheduling scheme. Background Art

[0002] For coal ports, a large number of transport vehicles enter the port every day. There are multiple coal chop in the port for storing different types of coal. Each coal chop has one or more working belts to transport the coal. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for determining a scheduling scheme.

[0004] According to one aspect of the present disclosure, a method for determining a scheduling scheme is provided, comprising:

[0005] Obtaining status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port;

[0006] Based on the state information, a target scheduling scheme including the corresponding relationship between the transport vehicle, the operation zone and the time is determined.

[0007] According to one aspect of the present disclosure, a device for determining a scheduling scheme is provided, comprising:

[0008] An information acquisition module, used to obtain status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port;

[0009] The scheme determination module is used to determine the target scheduling scheme including the corresponding relationship between the transport vehicle, the operation zone and the time based on the state information.

[0010] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned method for determining the scheduling scheme.

[0014] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned method for determining a scheduling scheme.

[0015] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned method for determining a scheduling scheme.

[0016] It can be seen from the above that in the solution provided by the embodiment of the present disclosure, the electronic device determines the target scheduling solution based on the status information, which significantly improves the efficiency of solution determination compared to the prior art in which the target scheduling solution is determined by experts.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0019] Figure 1 A coal port scene diagram provided by an embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram of a flow chart of a method for determining a first scheduling solution provided in an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of a flow chart of a method for determining a second scheduling solution provided in an embodiment of the present disclosure;

[0022] Figure 4 A flow chart of a method for generating an alternative scheduling solution provided in an embodiment of the present disclosure

[0023] Figure 5 A flowchart of a model training method provided in an embodiment of the present disclosure;

[0024] Figure 6 A schematic diagram of a flow chart of a method for determining a third scheduling solution provided in an embodiment of the present disclosure;

[0025] Figure 7 A schematic diagram of a flow chart of a method for determining a fourth scheduling solution provided in an embodiment of the present disclosure;

[0026] Figure 8 A schematic diagram of the structure of a device for determining a first scheduling solution provided in an embodiment of the present disclosure;

[0027] Fig. 9A schematic diagram of the structure of a device for determining a second scheduling solution provided in an embodiment of the present disclosure;

[0028] Fig.10 A schematic diagram of the structure of a module for determining an alternative scheduling solution provided in an embodiment of the present disclosure;

[0029] Fig.11 A schematic diagram of the structure of a model training module provided in an embodiment of the present disclosure;

[0030] Fig.12 A schematic diagram of the structure of a device for determining a third scheduling solution provided in an embodiment of the present disclosure;

[0031] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0034] First, combine Figure 1 The transport vehicle, the working belt and the coal chop involved in the embodiments of the present disclosure are explained.

[0035] Figure 1 A coal port scene diagram is shown. Figure 1 Among them, A, B, C are the coal blocks included in the coal port, a, b, c, d, e are the operating zones included in the coal port, and T1, T2, T3, ..., Tn are the transport vehicles arriving at the coal port.

[0036] Coal choppers are used to store coal. Different coal choppers may store the same or different types and capacities of coal.

[0037] The coal is transported to the corresponding coal chop through the operation belt. Specifically, the coal can be transported to different coal chops through different operation belts, or it can be transported to the same coal chop, for example: Figure 1 In the process, coal can be transported to the same coal chop A through working belts a and b.

[0038] The transport vehicle is used to transport coal to the coal port. The types and capacities of coal transported by different transport vehicles can be the same or different. After the transport vehicle transports the coal to the coal port, the coal transported by the transport vehicle needs to be unloaded onto the working belt, and the coal is transported to the corresponding coal chop through the working belt.

[0039] For large coal ports, there are hundreds of coal chop, hundreds of operation belts, and hundreds of transport vehicles arriving at the port every day. Different coal chop, different transport vehicles, and different operation belts have corresponding operation restrictions, so scheduling transport vehicles and operation belts is extremely complicated.

[0040] At present, the dispatching plans of coal ports are all temporarily planned and designated by experts with rich business experience. When there are more elements such as transport vehicles and coal chops, the experts' experience is often limited by local thinking.

[0041] The following describes a method for determining a scheduling scheme provided in an embodiment of the present disclosure.

[0042] See also Figure 2 , a flow chart of a method for determining a first scheduling solution is provided. The method includes the following steps S201-S202.

[0043] First, before describing the above steps, the execution subject of the embodiment of the present disclosure is described. The execution subject of the embodiment of the present disclosure may be an electronic device having a function of determining a scheduling scheme, and the electronic device may be a server, a cloud server, a terminal device connected to the Internet, etc.

[0044] Step S201: Obtain status information.

[0045] The above-mentioned status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicles arriving at the coal port.

[0046] The above three types of description information are explained below respectively.

[0047] The description information of the operation belt may include attribute information of the operation belt and coal information of the coal transported by the operation belt.

[0048] The attribute information of the operation belt may include: the identification of the operation belt, the size of the operation belt, the speed of the operation belt in transporting coal, the identification and quantity of the coal chops corresponding to the operation belt, whether the operation belt is in working state, etc.;

[0049] The coal information of the coal transported by the operation zone may include: the coal type of the transported coal, the maximum capacity of the transported coal, etc.

[0050] The description information of the coal chop may include attribute information of the coal chop and coal information of the coal stored in the coal chop.

[0051] The attribute information of the coal chop may include: the identification of the coal chop, the identification and number of the working belts connected to the coal chop, the maximum capacity of the coal chop, the available capacity of the coal chop at the start time, whether the coal chop is in working state, etc.;

[0052] The coal information of the stored coal may include: the coal type of the stored coal, the capacity of the coal stored in the coal chop, etc.

[0053] The description information of the transport vehicle may include attribute information of the transport vehicle and coal information of the coal unloaded by the transport vehicle.

[0054] The attribute information of the transport vehicle may include: the identification, size, maximum capacity, and the time when the transport vehicle arrives at the coal port.

[0055] The coal information of the coal to be unloaded may include: coal type, capacity of the occupied transport vehicle, etc.

[0056] Specifically, the status information can be obtained in the following two ways.

[0057] In one implementation, when obtaining the above-mentioned status information, a staff member may input the status information through a user interface provided by the electronic device, and the electronic device may thereby obtain the status information.

[0058] In another embodiment, description information of the working belt, the coal chop and the transport vehicle is stored in a preset database, and the electronic device can obtain the above three description information from the preset database as status information.

[0059] Step S202: Based on the status information, determine a target scheduling solution including the correspondence between the transport vehicle, the operation zone and the time.

[0060] The target scheduling scheme includes the correspondence between the transport vehicle, the operation zone and the time. In the subsequent scheduling, the time when the transport vehicle unloads the coal and the operation zone for transporting the unloaded coal can be determined based on the correspondence in the target scheduling scheme.

[0061] by Figure 1For example, when there is a corresponding relationship between the transport vehicle T1, the operating zone a and the time t, it means that the transport vehicle T1 unloads the coal at the time a and transports the coal unloaded by the transport vehicle through the operating zone a.

[0062] In one embodiment, each transport vehicle, each work belt and each moment can be randomly combined to obtain various possible schemes. Each possible scheme includes the corresponding relationship between the transport vehicles, work belts and moments. Based on the description information of the transport vehicles, work belts and the coal chops corresponding to the work belts included in each possible scheme, a scheme that meets the preset conditions is determined, and a scheme is randomly selected from the determined schemes to be determined as the target scheduling scheme.

[0063] The above-mentioned preset conditions may be set by the staff based on experience. For example, the preset conditions may include: the type of coal unloaded by the transport vehicle is the same as the type of coal transported by the working belt, the capacity of the coal unloaded by the transport vehicle is less than or equal to the maximum capacity of the coal transported by the working belt, the capacity of the coal unloaded by the transport vehicle is less than or equal to the maximum capacity of the coal chop corresponding to the working belt, etc.

[0064] Other implementation methods for determining the target scheduling scheme can be found in the following Figure 3 , Figure 6 The corresponding embodiments are not described in detail here.

[0065] It can be seen from the above that in the solution provided by the embodiment of the present disclosure, the electronic device determines the target scheduling solution based on the status information, which significantly improves the efficiency of solution determination compared to the prior art in which the target scheduling solution is determined by experts.

[0066] Furthermore, since the status information includes description information of the operating zone, description information of the coal chops, and description information of the transport vehicles arriving at the coal port, the above three types of description information are referred to when determining the target scheduling plan. Moreover, since the above three types of description information can comprehensively reflect the description information of the operating zone and coal chops included in the transport vehicles and the coal port, the target scheduling plan can be determined based on the more comprehensive description information, thereby improving the accuracy of the determined target scheduling plan.

[0067] If the status information in step S201 is the status information of the starting time, the starting time may be the time set by the staff. For example, the starting time may be set to 6:00 a.m. In this case, Figure 3 Steps S302-S303 of the illustrated embodiment are implemented. Figure 3 , Figure 3 A flowchart of a method for determining a second scheduling scheme provided in an embodiment of the present disclosure, the method includes the following steps S301-S303.

[0068] Step S301: Obtain status information.

[0069] The above-mentioned status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicles arriving at the coal port.

[0070] The above step S301 is the same as the above Figure 1 The step S201 of the illustrated embodiment is the same and will not be described in detail here.

[0071] Step S302: The state information at the starting moment is input into the information prediction model to be trained in a loop, and the state information at the starting moment is used as a training sample. The information prediction model is trained by reinforcement learning. When the end conditions of the model training are met, an alternative scheduling plan including the correspondence between the transport vehicle, the work belt and the time generated in each cycle is obtained.

[0072] In the process of cyclically inputting the state information at the starting time, the state information at the starting time is used as a training sample, and the information prediction model is trained by a reinforcement learning method. That is, in the process of cyclically inputting the state information at the starting time, the above information prediction model is trained in real time.

[0073] The above information prediction model can be an initial neural network model or a pre-trained neural network model.

[0074] The above-mentioned model training end condition may be pre-set by the staff, for example: the above-mentioned model training end condition may be reaching a preset number of iterations, information prediction model convergence, etc.

[0075] In the process of training the information prediction model, the model parameters of the information prediction model are constantly changing, so when the state information at the start time is input into the information prediction model in each cycle, the generated alternative scheduling schemes are different. After the model training end conditions are met, the alternative scheduling schemes generated in each cycle can be obtained.

[0076] For example: in the first cycle, the state information at the starting time is input into the information prediction model to be trained, and the first alternative scheduling plan episode 1 is obtained; if the end condition of model training is not met, the state information at the starting time is re-input into the information prediction model to be trained to obtain the second alternative scheduling plan episode 2. In this way, the cycle is repeated until the end condition of model training is met, and different alternative scheduling plans episode 1, episode 2, episode 3, ..., episode n can be obtained.

[0077] The process of inputting the state information at the start time into the information prediction model to obtain an alternative scheduling solution is called an iteration process. In an iteration process, the information prediction model will be trained multiple times. The training process of the above information prediction model can be found in Figure 5 The illustrated embodiment will not be described in detail here.

[0078] The generation process of the above alternative scheduling scheme can be found in the following Figure 4 The corresponding embodiments.

[0079] Step S303: Determine a target scheduling solution from the alternative scheduling solutions.

[0080] The above-mentioned alternative scheduling schemes include at least one alternative scheduling scheme. Based on this, in one implementation, when determining the target scheduling scheme, it can be implemented according to the following steps A1-A2.

[0081] Step A1: For each alternative scheduling scheme, based on the information of coal unloaded by the transport vehicle corresponding to each moment in the alternative scheduling scheme and the information of the coal chops connected to the corresponding operation belt, calculate the first reward value of the alternative scheduling scheme.

[0082] The first reward value reflects: the accuracy of determining the alternative scheduling scheme as the target scheduling scheme. When the first reward value is higher, it means that the alternative scheduling scheme is more accurately determined as the target scheduling scheme; when the first reward value is lower, it means that the alternative scheduling scheme is less accurately determined as the target scheduling scheme.

[0083] The information on the coal unloaded by the first transport vehicle may include: the coal capacity and coal type of the coal unloaded by the first transport vehicle, and the information on the coal chop connected to the first operation zone may include: the occupied capacity of the coal chop, information indicating whether coal has been stored, etc.

[0084] In the case where the coal capacity of the first transport vehicle included in the alternative scheduling scheme to unload coal is larger and the coal chops connected to the first operation zone are stored coal chops, the first reward value is higher. Based on this, in one implementation, the first reward value can be calculated according to the following formula:

[0085] r1=∑(weight-c1*new)

[0086] Among them, r1 represents the calculated first reward value, weight represents the coal capacity of the first transport vehicle included in the alternative scheduling plan to unload coal, c1 represents the first preset constant, and new represents the identification of whether the coal chop connected to the first operation belt has stored coal. If coal has been stored, new=1, if coal has not been stored, new=0.

[0087] Step A2: Determine a target scheduling solution from among the candidate scheduling solutions based on the first reward value of each candidate scheduling solution.

[0088] In one implementation, the alternative scheduling solution with the highest first reward value may be determined as the target scheduling solution.

[0089] Since the target scheduling scheme is determined from the alternative scheduling schemes based on the first reward value of each alternative scheduling scheme, and since the first reward value reflects the accuracy of determining the alternative scheduling scheme as the target scheduling scheme, when determining the target scheduling scheme, the accuracy corresponding to each alternative scheduling scheme is referred to, so that the accuracy of the determined target scheduling scheme is higher.

[0090] In the process of training the information prediction model, the model parameters of the information prediction model are constantly changing. Therefore, after each cycle inputs the state information at the starting time into the information prediction model, the generated alternative scheduling schemes are different, and the model parameters have tended to be stable in the later stage of training. The accuracy of the alternative scheduling schemes generated during this period is relatively high. When determining the scheduling scheme from multiple alternative scheduling schemes with higher accuracy generated by the training process, multiple schemes with higher accuracy can be referred to, thereby improving the accuracy of the determined scheduling scheme.

[0091] The alternative scheduling solutions generated in each iteration in the above step S302 can be as follows: Figure 4 The steps S401-S402 of the illustrated embodiment are determined. Figure 4 , Figure 4 A flowchart of a method for generating an alternative scheduling solution is provided, and the method includes the following steps S401-S402.

[0092] Step S401: Input the state information at the start time into the information prediction model to be trained, and obtain the first transport vehicle and the first operation zone corresponding to the start time output by the information prediction model. If the first cycle end condition is not met, execute step S402.

[0093] The first transport vehicle mentioned above is: a transport vehicle predicted by the information prediction model and used to unload coal at the starting time.

[0094] The first operating zone is: the operating zone predicted by the information prediction model and used to transport the coal unloaded by the first transport vehicle.

[0095] Step S402: Based on the first transport vehicle and the first working belt corresponding to the starting time, determine the status information of the next moment of the starting time, and update the starting time to the next moment of the starting time, return to step S401, until the end condition of the first loop is met, and obtain an alternative scheduling plan including the first transport vehicle, the first working belt, and the corresponding relationship between each moment.

[0096] The next moment after the start time is a moment after the start time and separated from the start time by a preset time. The preset time may be 5 minutes, 10 minutes, etc. For example, if the start time is 6:00am and the preset time is 10 minutes, the next moment after the start time is 6:10am.

[0097] The first cycle end condition may be preset by the staff. For example, the cycle end condition may be: obtaining the first transport vehicle and the first operating zone corresponding to the preset termination time, or obtaining the time and operating zone corresponding to all transport vehicles arriving at the coal port, or obtaining the transport vehicles and time corresponding to all operating zones.

[0098] In one embodiment, when determining the status information at the next moment, the description information of the first transport vehicle, the first operating belt and the coal chops corresponding to the first operating belt in the status information at the starting moment can be updated, and the updated status information can be determined as the status information at the next moment of the starting moment.

[0099] For example: Combine Figure 1 To illustrate, it is assumed that the status information at the starting moment includes the description information of coal chops A, B, and C, the description information of operating belts a, b, c, d, and e, and the description information of transport vehicles T1, T2, T3, ... Tn. If it is determined that the first transport vehicle corresponding to the starting moment is T1 and the first operating belt is a, then during subsequent scheduling, the coal in T1 will be unloaded to a at the starting moment and transferred to coal chop A through a. Therefore, after the starting moment, T1 changes from a never-unloaded state to an unloaded state, operating belt a changes from a never-occupied state to an occupied state, and the available capacity for storing coal in coal chop A decreases. Based on this, the description information of T1, a, and A in the status information at the starting moment can be updated.

[0100] Update the starting time to the next time after the starting time, and after returning to step S401, input the state information of the next time after the starting time into the information prediction model to be trained, and obtain the first transport vehicle and the first operating belt corresponding to the next time after the starting time output by the information prediction model. According to the above-mentioned loop method, until the end condition of the first loop is met, the first transport vehicle and the first operating belt corresponding to each time are obtained, thereby obtaining the correspondence between the first transport vehicle, the first operating belt, and each time, and determining the above-mentioned correspondence as an alternative scheduling scheme.

[0101] Since the first transport vehicle and the first operating belt are output by the information prediction model after the status information is input into the information prediction model in each cycle, the correspondence between the first transport vehicle, the first operating belt and the time targeted by the cycle can be determined. Therefore, after the end condition of the first cycle is met, an alternative scheduling plan including the correspondence between the time targeted by each cycle, the first transport vehicle and the first operating belt can be obtained.

[0102] Since the status information at each moment is related to the first transport vehicle and the first working belt corresponding to the previous moment, the status information of the next moment after the starting moment determined based on the first transport vehicle and the first working belt corresponding to the starting moment is highly accurate, so that the status information of each moment after the starting moment is determined with high accuracy, thereby improving the accuracy of the determined alternative scheduling scheme.

[0103] Above Figure 3 The information prediction model mentioned in the embodiment shown can be implemented as follows: Figure 5 Steps S501-S504 of the illustrated embodiment are used for training. Figure 5 , Figure 5 A flow chart of a model training method is provided, the method comprising the following steps S501-S504.

[0104] Step S501: Obtain a first predicted usefulness corresponding to the starting time output by the information prediction model.

[0105] In the above Figure 4 In step S401 of the illustrated embodiment, after the state information at the starting time is input into the information prediction model to be trained, the first transport vehicle and the first working belt corresponding to the starting time output by the information prediction model are obtained, and the information prediction model also outputs a first predicted usefulness, which is used to characterize the predicted usefulness of the first transport vehicle and the first working belt corresponding to the starting time.

[0106] Specifically, the usefulness corresponding to the starting time represents the usefulness of determining the first transport vehicle to unload coal and the first operating belt to transport coal at the starting time. The higher the usefulness, the better the usefulness of determining the first transport vehicle to unload coal and the first operating belt to transport coal at the starting time; the lower the usefulness, the worse the usefulness of determining the first transport vehicle to unload coal and the first operating belt to transport coal at the starting time.

[0107] The above-mentioned prediction helpfulness indicates the helpfulness of determining the first transport vehicle to unload coal and determining the first operating belt to transport coal at the starting moment, as predicted by the information prediction model.

[0108] Step S502: Obtain the first transport vehicle and the first operating zone corresponding to a preset number of consecutive target moments after the start moment output by the information prediction model.

[0109] The preset number can be set by the staff based on experience. For example, the preset number can be set to 16, 32, etc.

[0110] When the starting time is time t, the preset number is k, and the above target times include: t+1, t+2, ..., t+k+1.

[0111] In one implementation, the state information corresponding to each target moment may be obtained and input into the information prediction model to obtain the first transport vehicle and the first operating zone corresponding to each target moment.

[0112] Step S503: Calculate the actual usefulness of the first transport vehicle and the first operating belt corresponding to the starting time based on the information of the coal unloaded by the first transport vehicle corresponding to each target time and the information of the coal transported by the corresponding first operating belt.

[0113] The above-mentioned real beneficial degree represents the actual beneficial degree of determining the first transport vehicle to unload coal and determining the first operating belt to transport coal at the starting moment. The higher the real beneficial degree, the higher the actual beneficial degree of determining the first transport vehicle to unload coal and determining the first operating belt to transport coal at the starting moment. The lower the real beneficial degree, the lower the actual beneficial degree of determining the first transport vehicle to unload coal and determining the first operating belt to transport coal at the starting moment.

[0114] In the reinforcement learning process, the action performed at the current moment will affect the action performed at each moment after the current moment. The real helpfulness of the action performed at the current moment is related to the action performed at each moment after the current moment. Therefore, the real helpfulness corresponding to the starting moment can be calculated based on the above two types of information corresponding to each target moment.

[0115] In one embodiment of the present disclosure, the above step S503 may be implemented according to the following steps B1-B2.

[0116] Step B1: For each target moment, the second reward value of the target moment is calculated according to the information of the coal unloaded by the first transport vehicle corresponding to the target moment and the information of the coal chops connected to the corresponding first operating belt.

[0117] The second reward value reflects the accuracy of selecting the corresponding first transport vehicle to unload the coal and the corresponding first operating belt to transport the coal at the target time.

[0118] In one implementation, the second reward value may be calculated according to the following formula:

[0119] r2=w-c2*n

[0120] Among them, r2 represents the calculated second reward value, w represents the coal capacity of the first transport vehicle unloaded at the target time, c2 represents the second preset constant, and n represents the identification of whether the coal chop connected to the second operation belt has stored coal. If coal has been stored, n=1, and if coal has not been stored, n=0.

[0121] Step B2: According to the preset weights, weighted sum is performed on the second reward values ​​corresponding to each target moment, and the calculated sum is used as the actual usefulness of the first transport vehicle and the first operating zone corresponding to the starting moment.

[0122] In one implementation, the above-mentioned real benefit G can be calculated according to the following formula: t :

[0123]

[0124] Where k represents the preset number, γ represents the preset weight, γ∈[0,1], t represents the starting time, R t+k+1 Indicates the second reward value corresponding to the target time.

[0125] Since the real benefit of the first operation zone is calculated based on the second reward value corresponding to each target moment, the calculated real benefit is related to the second reward value corresponding to each target moment. Since the real benefit corresponding to the current moment is related to the reward values ​​corresponding to each moment after the current moment, the accuracy of the obtained real benefit is high, therefore, the accuracy of the calculated real benefit is high.

[0126] Step S504: Based on the first predicted helpfulness and the actual helpfulness corresponding to the starting time, adjust the model parameters of the information prediction model. If the model training end condition is not met, update the starting time to the next time of the starting time, and return to step S501 until the model training end condition is met to complete the training of the information prediction model.

[0127] In one implementation, the model parameters of the information prediction model may be adjusted in step S504 according to the following steps C1-C3.

[0128] Step C1: Calculate the difference between the first predicted helpfulness corresponding to the starting time and the actual helpfulness.

[0129] When calculating the above difference, the difference between the actual beneficialness and the first predicted beneficialness may be calculated, or the square of the difference between the actual beneficialness and the first predicted beneficialness may be calculated.

[0130] Step C2: Based on the calculated difference, determine the loss value of the information prediction model.

[0131] In one implementation, the loss value L of the information prediction model can be calculated as follows: critic :

[0132] L critic =E[(G t -V π (s)) 2 ]

[0133] Among them, E[·] represents the mathematical expectation, G t Indicates the real benefit corresponding to the starting time, V π (s) represents the first predicted helpfulness corresponding to the starting time.

[0134] Step C3: Based on the loss value, adjust the model parameters of the information prediction model.

[0135] When adjusting the model parameters, a preset model parameter adjustment algorithm may be used for adjustment. The model parameter adjustment algorithm may be GD (Gradient Descent), SGD (Stochastic Gradient Descent), or the like.

[0136] From the above, it can be seen that since the difference between the first predicted beneficialness and the actual beneficialness reflects the accuracy of the prediction ability of the information prediction model, the loss value of the information prediction model is determined based on the above difference, and the model parameters of the information prediction model are adjusted based on the above loss value. This can improve the information prediction accuracy of the information prediction model after the adjustment of the parameters.

[0137] From the above, it can be seen that since the actual beneficialness reflects the actual beneficialness of the first transport vehicle and the first working belt corresponding to the starting moment, the first predicted beneficialness is the predicted beneficialness of the first transport vehicle and the first working belt corresponding to the starting moment predicted by the information prediction model, and adjusting the model parameters of the information prediction model based on the first predicted beneficialness and the actual beneficialness can improve the information prediction ability of the information prediction model.

[0138] In the aforementioned Figure 2 In the step S201 of the embodiment shown, when the state information is the state information at the start time, in addition to Figure 3 The steps of the embodiment shown in the figure can also be implemented according to Figure 6 In the embodiment shown, steps S602-S603 are implemented. Figure 6 , Figure 6 A flowchart of a method for determining a third scheduling solution provided in an embodiment of the present disclosure. The method includes the following steps S601-S603.

[0139] Step S601: Obtain status information.

[0140] The above-mentioned status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicles arriving at the coal port.

[0141] The above step S601 is similar to the above step S602. Figure 1 The step S201 of the illustrated embodiment is the same and will not be described in detail here.

[0142] Step S602: Based on the state information at the start time, determine the correspondence between the start time, the second transport vehicle, and the second operation zone. If the second cycle end condition is not met, execute step S603.

[0143] In one implementation, the state information at the starting time may be input into a pre-trained information prediction model to obtain the second transport vehicle and the second operating zone output by the information prediction model, thereby obtaining the corresponding relationship between the starting time, the second transport vehicle, and the second operating zone.

[0144] The information prediction model is used to predict the transport vehicle for unloading coal and the operation belt for transporting the unloaded coal based on the status information.

[0145] Indeed, other implementations of the above-mentioned target transport vehicle and target operation belt can be found in the subsequent Figure 7 The corresponding embodiments are not described in detail here.

[0146] Step S603: Based on the second transport vehicle and the second work belt corresponding to the starting time, determine the status information of the next moment of the starting time, update the starting time to the next moment of the starting time, and return to step S602 until the second loop end condition is met, and obtain the target scheduling plan including the corresponding relationship between each moment, the second work belt, and the second transport vehicle.

[0147] The second cycle end condition may be preset by the staff. For example, the second cycle end condition may be: determining the second transport vehicle and the second operating zone corresponding to the preset end time, or determining the time and the second operating zone corresponding to each transport vehicle, or determining the time and the second transport vehicle corresponding to each operating zone.

[0148] After determining the second transport vehicle and the second operating zone corresponding to the starting time, if the second cycle end condition is not met, it is necessary to continue to determine the next moment of the starting time, the corresponding relationship between the transport vehicle and the operating zone until the above cycle end condition is met to obtain the target scheduling plan.

[0149] For example, the starting time is t, and each time after the starting time is t+1, t+2, t+3, ..., t+n. Based on the state information S at the starting time t, t , determine the corresponding relationship between the starting time t, the second transport vehicle and the second operating zone, and when the second cycle end condition is not met, based on a t , get the state information S at time t+1 t+1 , and return to step S202, determine the correspondence between time t+1, the second target transport vehicle, and the second target operating zone, and follow this loop method until the second loop end condition is met to obtain a target scheduling plan including the correspondence between each time, the second transport vehicle, and the second operating zone.

[0150] From the above, it can be seen that based on the status information at the starting time, the correspondence between the starting time, the second transport vehicle, and the second working zone is determined. When the end condition of the second cycle is not met, the correspondence between each moment after the starting time, the second transport vehicle, and the second working zone is determined cyclically, so as to obtain the target scheduling plan. Compared with the existing technology, there is no need for experts to determine the scheduling plan based on experience, which improves the efficiency of determining the target scheduling plan.

[0151] In addition, since the status information at each moment is related to the transport vehicle that unloaded the coal at the previous moment and the work belt that transported the coal, the accuracy of the status information of the next moment of the starting moment determined based on the second transport vehicle and the second work belt corresponding to the starting moment is high, thereby improving the accuracy of the corresponding relationship determined based on the status information, and further improving the accuracy of the target scheduling plan.

[0152] In the above Figure 6 When determining the corresponding relationship in step S602 of the embodiment shown in the figure, the following steps can be used: Figure 7 In the embodiment shown, steps S702-S704 are implemented. Figure 7 , Figure 7 A flowchart of a fourth method for determining a scheduling solution provided in an embodiment of the present disclosure. The method includes the following steps S701-S705.

[0153] Step S701: Obtain status information.

[0154] The above-mentioned status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicles arriving at the coal port.

[0155] The above step S701 is similar to the above step S702. Figure 1 The step S201 of the illustrated embodiment is the same and will not be described in detail here.

[0156] Step S702: Based on the status information at the starting time, determine the transport vehicle used to unload the coal at the starting time as the second transport vehicle.

[0157] In one embodiment of the present disclosure, the above step S702 may be implemented according to the following steps D1-D2.

[0158] Step D1: Based on the state information at the starting time, predict a second predicted usefulness of each transport vehicle as a transport vehicle for unloading coal at the starting time.

[0159] The second predicted helpfulness represents the helpfulness of determining each transport vehicle as a transport vehicle for unloading coal at the initial moment.

[0160] In one embodiment, when predicting the second predicted helpfulness, the state information at the starting time may be input into a pre-trained helpfulness prediction model to obtain the second predicted helpfulness corresponding to each transport vehicle output by the model.

[0161] Step D2: Based on the second predicted usefulness corresponding to each transport vehicle, determine the transport vehicle for unloading coal at the starting time from among the transport vehicles.

[0162] When determining the target transport vehicle, the transport vehicle with the second largest predicted usefulness may be selected as the transport vehicle for unloading the coal at the starting moment.

[0163] Since the second predicted helpfulness can characterize the helpfulness of each transport vehicle as a transport vehicle for unloading coal at the starting moment, when the second transport vehicle is determined based on the second predicted helpfulness, the determined transport vehicle can be determined with high accuracy as a transport vehicle for unloading coal at the starting moment.

[0164] Step S703: Based on the status information at the starting time and the corresponding second transport vehicle, determine the operating zone used for transporting coal at the starting time as the second operating zone.

[0165] Since the second operating zone needs to transport the coal unloaded by the second transport vehicle, the second operating zone corresponding to the starting moment is not only related to the status information at the starting moment, but also related to the second transport vehicle corresponding to the starting moment. Therefore, it is necessary to determine the second operating zone corresponding to the starting moment based on the status information at the starting moment and the corresponding second transport vehicle.

[0166] In one embodiment of the present disclosure, the above step S703 may be implemented according to the following steps E1-E4.

[0167] Step E1: Based on the state information at the starting time, predict the third predicted usefulness of each operating zone as an operating zone for transporting coal at the starting time.

[0168] The third predicted helpfulness represents the helpfulness of determining each operating zone as an operating zone for transporting coal at the starting moment.

[0169] When the third predicted beneficialness is predicted, the state information at the starting time may be input into the beneficialness prediction model to obtain the third predicted beneficialness corresponding to each operation band output by the beneficialness prediction model.

[0170] Step E2: for each operation zone, determine the matching degree between the description information of the operation zone and the description information of the second transport vehicle corresponding to the starting time.

[0171] Since the more the operation belt and the transport vehicle match, the better the operation belt can transport the coal unloaded by the transport vehicle, the above matching degree can reflect the possibility that each operation belt can transport the coal unloaded by the second transport vehicle. The higher the matching degree, the greater the possibility; the lower the matching degree, the smaller the possibility.

[0172] Specifically, the description information of the second transport vehicle may include the size of the second transport vehicle, the type of coal unloaded by the second transport vehicle, the coal capacity, etc.

[0173] Since the coal type, coal capacity and other coal information of each operation belt are different, and the size, capacity and other operation belt information of each operation belt are also different, the possibility that different operation belts can transport the coal unloaded by the target transport vehicle is also different. Therefore, it is necessary to calculate the matching degree reflecting the above possibilities.

[0174] Specifically, when calculating the matching degree, the number of identical information in the description information of the operation belt and the description information of the second transport vehicle may be determined, and based on the corresponding relationship between the matching degree and the number, the matching degree corresponding to the number may be determined.

[0175] Step E3: Based on the calculated matching degree, determine the candidate operation belt from the operation belts.

[0176] When determining the candidate operation belts, the operation belts with a matching degree greater than a matching degree threshold may be determined as the candidate operation belts, or a preset number of operation belts with the highest matching degrees may be determined as the candidate operation belts.

[0177] Step E4: Based on the third predicted usefulness corresponding to the candidate operating zones, an operating zone for transporting coal is determined from among the candidate operating zones as the second operating zone.

[0178] In one implementation, the candidate operating zone with the highest third predicted usefulness may be determined as the operating zone for transporting coal.

[0179] Since the matching degree between the description information of the operating belt and the description information of the second transport vehicle can reflect the matching degree between the operating belt and the second transport vehicle, the higher the matching degree, the higher the possibility that the operating belt can transport the coal unloaded by the second transport vehicle, and the lower the matching degree, the lower the possibility that the operating belt can transport the coal unloaded by the second transport vehicle. Therefore, the alternative operating belt determined based on the above matching degree has a higher possibility of transporting the coal unloaded by the target transport vehicle, and then based on the third predicted beneficial degree reflecting the beneficial degree corresponding to each alternative operating belt, the operating belt used to transport coal can be determined more accurately.

[0180] Step S704: Obtain the correspondence between the starting time, the second transport vehicle and the second operation zone.

[0181] Step S705: If the second loop end condition is not met, determine the status information of the next moment of the starting time based on the second transport vehicle and the second work belt corresponding to the starting time, update the starting time to the next moment of the starting time, and return to step S702 until the second loop end condition is met, and obtain the target scheduling plan including the corresponding relationship between each moment, the second work belt, and the second transport vehicle.

[0182] In one embodiment, when the status information of the next moment after the starting moment is obtained, the description information of the second transport vehicle, the second operating belt and the coal chops connected to the second operating belt in the status information of the starting moment is updated as the status information of the next moment after the starting moment.

[0183] From the above, it can be seen that the second operating belt corresponding to the starting moment is determined based on the status information at the starting moment and the corresponding second transport vehicle, so that the determined second operating belt is related to the second transport vehicle. Since the second operating belt is related to the second transport vehicle, it reflects that the second operating belt can be more suitable for transporting the coal unloaded by the second transport vehicle. Therefore, the second operating belt determined by the scheme of the embodiment of the present disclosure has high accuracy.

[0184] Corresponding to the above-mentioned method for determining a scheduling scheme, the embodiment of the present disclosure also provides a device for determining a scheduling scheme.

[0185] See also Figure 8 , Figure 8 A schematic diagram of the structure of a device for determining a first scheduling scheme provided in an embodiment of the present disclosure, wherein the device includes the following modules 801-802.

[0186] The information acquisition module 801 is used to obtain status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port;

[0187] The plan determination module 802 is used to determine a target scheduling plan including the corresponding relationship between the transport vehicle, the operation zone and the time based on the state information.

[0188] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a device for determining a second scheduling scheme provided in an embodiment of the present disclosure, wherein the device includes the following modules 901-903.

[0189] The information acquisition module 901 is used to obtain status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port;

[0190] The first scheme determination submodule 902 is used to cyclically input the state information at the starting time into the information prediction model to be trained, use the state information at the starting time as a training sample, and train the information prediction model in a reinforcement learning manner. When the model training end condition is met, an alternative scheduling scheme including the corresponding relationship between the transport vehicle, the operation belt, and the time generated in each cycle is obtained;

[0191] The second solution determination submodule 903 is used to determine a target scheduling solution from the alternative scheduling solutions.

[0192] In one embodiment of the present disclosure, the above-mentioned alternative scheduling schemes include at least one alternative scheduling scheme, and the above-mentioned second scheme determination submodule 903 is specifically used to calculate, for each alternative scheduling scheme, the first reward value of the alternative scheduling scheme based on the information of the coal unloaded by the transport vehicle corresponding to each moment in the alternative scheduling scheme, and the information of the coal chops connected to the corresponding work belt; based on the first reward value of each alternative scheduling scheme, determine the target scheduling scheme from each alternative scheduling scheme.

[0193] See also Fig.10 , Fig.10 A schematic diagram of the structure of a module for determining an alternative scheduling solution provided in an embodiment of the present disclosure, wherein the module includes the following units 1001-1002.

[0194] The information acquisition unit 1001 is used to input the state information at the starting time into the information prediction model to be trained, and obtain the first transport vehicle and the first operation belt corresponding to the starting time output by the information prediction model, wherein the first transport vehicle is: the transport vehicle predicted by the information prediction model and used to unload the coal at the starting time, and the first operation belt is: the operation belt predicted by the information prediction model and used to transport the coal unloaded by the first transport vehicle;

[0195] The plan determination unit 1002 is used to determine the state information of the next moment of the starting time based on the first transport vehicle and the first working belt corresponding to the starting time if the first cycle end condition is not met, and update the starting time to the next moment of the starting time, triggering the information acquisition unit 1001 until the first cycle end condition is met, and obtaining an alternative scheduling plan including the first transport vehicle, the first working belt, and the corresponding relationship between each moment.

[0196] See also Fig.11 , Fig.11 A structural diagram of a model training module provided in an embodiment of the present disclosure, the module includes the following sub-modules 1101-1104.

[0197] The usefulness obtaining submodule 1101 is used to obtain the first predicted usefulness corresponding to the starting time output by the information prediction model, wherein the first predicted usefulness is used to characterize the predicted usefulness of the first transport vehicle and the first operating zone corresponding to the starting time;

[0198] An information acquisition submodule 1102 is used to obtain the first transport vehicle and the first operation zone corresponding to a preset number of consecutive target moments after the starting moment output by the information prediction model;

[0199] The usefulness calculation submodule 1103 is used to calculate the real usefulness of the first transport vehicle and the first operating belt corresponding to the starting time according to the information of the coal unloaded by the first transport vehicle corresponding to each target time and the information of the coal transported by the corresponding first operating belt;

[0200] The parameter adjustment submodule 1104 is used to adjust the model parameters of the information prediction model based on the first predicted helpfulness corresponding to the starting moment and the actual helpfulness. If the model training end condition is not met, the starting moment is updated to the next moment of the starting moment, and the helpfulness acquisition submodule 1101 is triggered until the model training end condition is met, thereby realizing the training of the information prediction model.

[0201] In one embodiment of the present disclosure, the above-mentioned helpfulness calculation submodule 1103 is specifically used to calculate, for each target moment, the second reward value of the target moment according to the information of the coal unloaded by the first transport vehicle corresponding to the target moment, and the information of the coal chops connected to the corresponding first operating belt; the second reward value corresponding to each target moment is weightedly summed according to the preset weights, and the calculated sum is used as the true helpfulness of the first transport vehicle and the first operating belt corresponding to the starting moment.

[0202] In one embodiment of the present disclosure, the above-mentioned parameter adjustment submodule 1104 is specifically used to calculate the difference between the first predicted beneficialness corresponding to the starting moment and the actual beneficialness; based on the calculated difference, determine the loss value of the information prediction model; based on the loss value, adjust the model parameters of the information prediction model.

[0203] See also Fig.12 , Fig.12 A schematic diagram of the structure of a device for determining a third scheduling scheme provided in an embodiment of the present disclosure, wherein the device includes the following modules 1201-1203.

[0204] The information acquisition module 1201 is used to obtain status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port;

[0205] A corresponding relationship determination submodule 1202 is used to determine the corresponding relationship between the starting time, the second transport vehicle, and the second operation zone based on the state information at the starting time;

[0206] The third scheme determination submodule 1203 is used to determine the state information of the next moment of the starting moment based on the second transport vehicle and the second work belt corresponding to the starting moment if the second cycle end condition is not met, and update the starting moment to the next moment of the starting moment, triggering the corresponding relationship determination submodule 1202 until the second cycle end condition is met, thereby obtaining a target scheduling scheme including the corresponding relationship between each moment, the second work belt, and the second transport vehicle.

[0207] In one embodiment of the present disclosure, the above-mentioned corresponding relationship determination submodule 1202 includes:

[0208] a transport vehicle determining unit, configured to determine, based on the state information at the starting time, a transport vehicle used for unloading the coal at the starting time as the second transport vehicle;

[0209] an operating zone determining unit, configured to determine, based on the state information at the starting time and the corresponding second transport vehicle, an operating zone for transporting coal at the starting time as the second operating zone;

[0210] The corresponding relationship determining unit is used to obtain the corresponding relationship between the starting time, the second transport vehicle and the second operating zone.

[0211] In one embodiment of the present disclosure, the above-mentioned transport vehicle determination unit is specifically used to predict the second predicted helpfulness of each transport vehicle as a transport vehicle for unloading coal at the starting moment based on the status information at the starting moment; and determine the transport vehicle for unloading coal at the starting moment from among the transport vehicles based on the second predicted helpfulness corresponding to each transport vehicle.

[0212] In one embodiment of the present disclosure, the above-mentioned operation belt determination unit is specifically used to predict the third predicted usefulness of each operation belt as an operation belt for transporting coal at the starting moment based on the state information at the starting moment; for each operation belt, determine the matching degree between the description information of the operation belt and the description information of the second transport vehicle corresponding to the starting moment; based on the calculated matching degree, determine an alternative operation belt from the operation belts; based on the third predicted usefulness corresponding to the alternative operation belts, determine an operation belt for transporting coal from each alternative operation belt as the second operation belt.

[0213] It can be seen from the above that in the solution provided by the embodiment of the present disclosure, the electronic device determines the target scheduling solution based on the status information, which significantly improves the efficiency of solution determination compared to the prior art in which the target scheduling solution is determined by experts.

[0214] Furthermore, since the status information includes description information of the operating zone, description information of the coal chops, and description information of the transport vehicles arriving at the coal port, the above three types of description information are referred to when determining the target scheduling plan. Moreover, since the above three types of description information can comprehensively reflect the description information of the operating zone and coal chops included in the transport vehicles and the coal port, the target scheduling plan can be determined based on the more comprehensive description information, thereby improving the accuracy of the determined target scheduling plan.

[0215] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0216] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0217] An embodiment of the present disclosure provides an electronic device, including:

[0218] at least one processor; and

[0219] a memory communicatively connected to the at least one processor; wherein,

[0220] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the aforementioned method for determining the scheduling scheme.

[0221] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the aforementioned method for determining a scheduling scheme.

[0222] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements the aforementioned method for determining a scheduling scheme when executed by a processor.

[0223] It can be seen from the above that in the solution provided by the embodiment of the present disclosure, the electronic device determines the target scheduling solution based on the status information, which significantly improves the efficiency of solution determination compared to the prior art in which the target scheduling solution is determined by experts.

[0224] Furthermore, since the status information includes description information of the operating zone, description information of the coal chops, and description information of the transport vehicles arriving at the coal port, the above three types of description information are referred to when determining the target scheduling plan. Moreover, since the above three types of description information can comprehensively reflect the description information of the operating zone and coal chops included in the transport vehicles and the coal port, the target scheduling plan can be determined based on the more comprehensive description information, thereby improving the accuracy of the determined target scheduling plan.

[0225] Fig.13 A schematic block diagram of an example electronic device 1300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0226] like Fig.13As shown, the device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0227] A number of components in the device 1300 are connected to the I / O interface 1305, including: an input unit 1306, such as a keyboard, a mouse, etc.; an output unit 1307, such as various types of displays, speakers, etc.; a storage unit 1308, such as a disk, an optical disk, etc.; and a communication unit 1309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0228] The computing unit 1301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1301 performs the various methods and processes described above, such as a method for determining a scheduling scheme. For example, in some embodiments, the method for determining a scheduling scheme may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1300 via the ROM 1302 and / or the communication unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the method for determining the scheduling scheme described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to execute the scheduling scheme determination method in any other appropriate manner (for example, by means of firmware).

[0229] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0230] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0231] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0232] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0233] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0234] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0235] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0236] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for determining a scheduling scheme, comprising: Obtaining status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port; Based on the state information, determining a target scheduling scheme including corresponding relationships between transport vehicles, operation zones and time; Wherein, when the state information is the state information at the starting time, the target scheduling scheme including the correspondence between the transport vehicle, the operation zone and the time is determined based on the state information, including: The state information at the starting time is cyclically input into the information prediction model to be trained, and the state information at the starting time is used as a training sample. The information prediction model is trained by a reinforcement learning method. When the model training end condition is met, an alternative scheduling scheme including the corresponding relationship between the transport vehicle, the operation belt and the time generated in each cycle is obtained; Determine the target scheduling plan from the alternative scheduling plans; The information prediction model is trained as follows: Obtaining a first predicted usefulness corresponding to the starting time output by the information prediction model, wherein the first predicted usefulness is used to characterize the predicted usefulness of the first transport vehicle and the first operating zone corresponding to the starting time; Obtaining the first transport vehicle and the first operating zone corresponding to a preset number of consecutive target moments after the starting moment output by the information prediction model; Calculate the real usefulness of the first transport vehicle and the first operating belt corresponding to the starting time according to the information of the coal unloaded by the first transport vehicle corresponding to each target time and the information of the coal transported by the first operating belt corresponding to the target time; Based on the first predicted helpfulness corresponding to the starting moment and the actual helpfulness, the model parameters of the information prediction model are adjusted. If the model training end condition is not met, the starting moment is updated to the next moment of the starting moment, and the step of obtaining the first predicted helpfulness corresponding to the starting moment output by the information prediction model is returned to until the model training end condition is met, thereby completing the training of the information prediction model.

2. The method according to claim 1, wherein: The alternative scheduling scheme includes at least one alternative scheduling scheme, and determining the target scheduling scheme from the alternative scheduling schemes includes: For each alternative scheduling scheme, based on the information of coal unloaded by the transport vehicle corresponding to each moment in the alternative scheduling scheme and the information of the coal chops connected to the corresponding operation belt, a first reward value of the alternative scheduling scheme is calculated; A target scheduling solution is determined from among the candidate scheduling solutions based on the first reward value of each candidate scheduling solution.

3. The method according to claim 1, wherein: Generate alternative scheduling plans as follows: Input the state information at the starting time into the information prediction model to be trained, and obtain the first transport vehicle and the first operating zone corresponding to the starting time output by the information prediction model, wherein the first transport vehicle is: the transport vehicle predicted by the information prediction model and used to unload the coal at the starting time, and the first operating zone is: the operating zone predicted by the information prediction model and used to transport the coal unloaded by the first transport vehicle; If the first cycle end condition is not met, the state information of the next moment of the starting moment is determined based on the first transport vehicle and the first work belt corresponding to the starting moment, and the starting moment is updated to the next moment of the starting moment, and the process returns to the step of inputting the state information of the starting moment into the information prediction model to be trained until the first cycle end condition is met, and an alternative scheduling scheme including the first transport vehicle, the first work belt, and the corresponding relationship between each moment is obtained.

4. The method according to claim 1, wherein: The calculation of the real usefulness of the first transport vehicle and the first operating zone corresponding to each target moment based on the information of the coal unloaded by the first transport vehicle and the information of the coal transported by the corresponding first operating zone, which is used to characterize the real usefulness of the first transport vehicle and the first operating zone corresponding to the starting moment, includes: For each target moment, the second reward value of the target moment is calculated according to the information of the coal unloaded by the first transport vehicle corresponding to the target moment and the information of the coal chops connected to the corresponding first operation belt; According to the preset weights, the second reward values ​​corresponding to each target moment are weighted and summed, and the calculated sum is used as the true usefulness of the first transport vehicle and the first operating belt corresponding to the starting moment.

5. The method according to claim 1, wherein: The adjusting the model parameters of the information prediction model based on the first predicted usefulness and the actual usefulness corresponding to the starting time includes: Calculating the difference between the first predicted helpfulness corresponding to the starting moment and the actual helpfulness; Determining a loss value of the information prediction model based on the calculated difference; Based on the loss value, the model parameters of the information prediction model are adjusted.

6. The method according to claim 1, wherein: When the state information is state information at the starting time, determining the target scheduling scheme including the corresponding relationship between the transport vehicle, the operation zone and the time based on the state information includes: Based on the state information at the starting time, determining a corresponding relationship between the starting time, the second transport vehicle, and the second operating zone; If the second cycle end condition is not met, based on the second transport vehicle and the second work belt corresponding to the starting time, determine the status information of the next moment of the starting time, update the starting time to the next moment of the starting time, and return to the step of determining the correspondence between the starting time, the second transport vehicle, and the second work belt based on the status information of the starting time until the second cycle end condition is met, and obtain the target scheduling plan including the correspondence between each moment, the second work belt, and the second transport vehicle.

7. The method according to claim 6, wherein: The determining of the correspondence between the starting time, the second transport vehicle, and the second operating zone based on the state information at the starting time includes: Based on the state information at the starting time, determining a transport vehicle used to unload the coal at the starting time as the second transport vehicle; Based on the state information at the starting time and the corresponding second transport vehicle, determining the operating zone for transporting coal at the starting time as the second operating zone; The corresponding relationship between the starting time, the second transport vehicle and the second operating zone is obtained.

8. The method according to claim 7, wherein: The determining, based on the state information at the starting time, a transport vehicle for unloading coal at the starting time comprises: Based on the state information at the starting time, predicting a second predicted usefulness of each transport vehicle as a transport vehicle for unloading coal at the starting time; Based on the second predicted usefulness corresponding to each transport vehicle, a transport vehicle for unloading the coal at the starting time is determined from among the transport vehicles.

9. The method according to claim 7, wherein: The step of determining the operation zone for transporting coal at the starting time as the second operation zone based on the state information at the starting time and the corresponding second transport vehicle includes: Based on the state information at the starting time, predicting a third predicted usefulness of each operation zone as an operation zone for transporting coal at the starting time; For each operation zone, determining a matching degree between the description information of the operation zone and the description information of the second transport vehicle corresponding to the starting time; Based on the calculated matching degree, determine the candidate operation belt from the operation belt; Based on the third predicted usefulness corresponding to the candidate operating zones, an operating zone for transporting coal is determined from among the candidate operating zones as the second operating zone.

10. A device for determining a scheduling scheme, comprising: An information acquisition module, used to obtain status information, wherein the status information includes: description information of the operation zone included in the coal port, description information of the included coal chops, and description information of the transport vehicle arriving at the coal port; A scheme determination module, used to determine a target scheduling scheme including a correspondence between transport vehicles, operation zones and time based on the state information; Wherein, when the state information is the state information at the starting time, the solution determination module includes: The first scheme determination submodule is used to cyclically input the state information at the starting time into the information prediction model to be trained, use the state information at the starting time as a training sample, and train the information prediction model by a reinforcement learning method. When the model training end condition is met, an alternative scheduling scheme including the corresponding relationship between the transport vehicle, the operation belt, and the time generated in each cycle is obtained; The second scheme determination submodule is used to determine a target scheduling scheme from the alternative scheduling schemes; The device also includes a model training module, and the model training module includes: A usefulness obtaining submodule, used to obtain a first predicted usefulness corresponding to the starting time output by the information prediction model, wherein the first predicted usefulness is used to characterize the predicted usefulness of the first transport vehicle and the first operating zone corresponding to the starting time; An information acquisition submodule, used for obtaining the first transport vehicle and the first operation zone corresponding to a preset number of consecutive target moments after the starting moment output by the information prediction model; The usefulness calculation submodule is used to calculate the real usefulness of the first transport vehicle and the first operating belt corresponding to the starting time according to the information of the coal unloaded by the first transport vehicle corresponding to each target time and the information of the coal transported by the corresponding first operating belt; The parameter adjustment submodule is used to adjust the model parameters of the information prediction model based on the first predicted helpfulness and the actual helpfulness corresponding to the starting moment. If the model training end condition is not met, the starting moment is updated to the next moment of the starting moment, and the helpfulness acquisition submodule is triggered until the model training end condition is met, thereby realizing the training of the information prediction model.

11. The device according to claim 10, wherein: The alternative scheduling scheme includes at least one alternative scheduling scheme, and the second scheme determination submodule is specifically used to calculate, for each alternative scheduling scheme, a first reward value of the alternative scheduling scheme based on information of coal unloaded by a transport vehicle corresponding to each moment in the alternative scheduling scheme and information of coal chops connected to the corresponding operation belt; A target scheduling solution is determined from among the candidate scheduling solutions based on the first reward value of each candidate scheduling solution.

12. The device according to claim 10, wherein: The first solution determines a submodule, including: An information acquisition unit is used to input the state information at the starting time into the information prediction model to be trained, and obtain the first transport vehicle and the first operating belt corresponding to the starting time output by the information prediction model, wherein the first transport vehicle is: the transport vehicle predicted by the information prediction model and used to unload the coal at the starting time, and the first operating belt is: the operating belt predicted by the information prediction model and used to transport the coal unloaded by the first transport vehicle; A plan determination unit is used to determine the state information of the next moment of the starting time based on the first transport vehicle and the first working belt corresponding to the starting time if the first cycle end condition is not met, update the starting time to the next moment of the starting time, and trigger the information acquisition unit until the first cycle end condition is met, thereby obtaining an alternative scheduling plan including the first transport vehicle, the first working belt, and the corresponding relationship between each moment.

13. The device according to claim 10, wherein: When the state information is state information at the starting time, the solution determination module includes: A corresponding relationship determination submodule, used to determine the corresponding relationship between the starting time, the second transport vehicle, and the second operating zone based on the state information at the starting time; The third scheme determination submodule is used to determine the status information of the next moment of the starting time based on the second transport vehicle and the second work belt corresponding to the starting time if the second cycle end condition is not met, and update the starting time to the next moment of the starting time, triggering the corresponding relationship determination submodule until the second cycle end condition is met, thereby obtaining a target scheduling scheme including the corresponding relationship between each moment, the second work belt, and the second transport vehicle.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

16. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images