Railway precast beam yard beam storage scheduling optimization method and device based on reinforcement learning

Abstract

The invention provides a railway precast beam yard beam storage scheduling optimization method and device based on reinforcement learning, and relates to the technical field of railway bridge construction engineering management.The method comprises the steps that a pre-established simulation model of a target beam yard is obtained; wherein the simulation model comprises operation behavior parameters of the target beam field, and the operation behavior parameters are used for representing behaviors and interaction modes of all modules of the target beam field; and based on a predetermined objective function, using a deep neural network based on reinforcement learning to optimize the operation behavior parameters so as to optimize the beam storage schedule of the target beam field based on the optimized operation behavior parameters. According to the method provided by the invention, a corresponding simulation model is established, model parameters are convenient to adjust, and interaction with the working condition of a beam field can be more closely realized; a deep neural network based on reinforcement learning is used for adjusting model behaviors, and the beam storage plan scheduling optimization technology under the limitation of multiple resources such as pedestals, equipment and personnel is broken through.

Description

Technical Field

[0001] The present invention relates to the technical field of railway bridge construction project management, and in particular to a method and device for optimizing the scheduling of railway prefabricated beams in a beam storage yard based on reinforcement learning. Background Art

[0002] Precast beam yards require a lot of resources and occupy a large area, and they urgently need to improve the efficiency of high-speed railway bridge construction through intelligent and efficient management. With the widespread application of engineering construction management platforms, the production scheduling of precast beam yards has begun to be managed in an informationized manner.

[0003] Specifically in terms of beam storage scheduling, the current mainstream beam storage scheduling decisions and plans are usually highly subjective, mainly based on the production experience of project engineers and some relevant standard documents and other materials, lacking data-supported quantitative analysis and intelligent simulation optimization methods. In recent years, in response to the problems of frequent conflicts in the scheduling of beam storage in traditional beam yards, insufficient site utilization, and long scheduling time, some scheduling plans based on group optimization algorithms have emerged, but there are problems such as a single optimization goal and a loose connection with the on-site construction process, making it difficult to cope with changes in on-site uncertainties.

[0004] How to optimize the storage scheduling of railway precast beam yards and improve the optimization efficiency is a technical problem that needs to be solved at present. Summary of the invention

[0005] The present invention provides a method and device for optimizing the scheduling of railway prefabricated beams in a beam yard based on reinforcement learning, so as to solve the defects existing in the prior art.

[0006] The present invention provides a method for optimizing the scheduling of railway prefabricated beams in a storage yard based on reinforcement learning, comprising the following steps: Acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0007] According to a method for optimizing the storage scheduling of railway prefabricated beam yards based on reinforcement learning provided by the present invention, before obtaining a pre-established simulation model of a target beam yard, the method further includes: Acquire a pre-established business logic model of a target beam yard, and perform simulation operation logic construction on the business logic model to obtain a simulation model of the target beam yard; Among them, the business logic model includes a beam manufacturing area business logic model and a beam storage area business logic model; the beam manufacturing area business logic model is used to represent the business status of the beam manufacturing area in the target beam yard, and the beam storage area business logic model is used to represent the construction status of the beam storage area in the target beam yard.

[0008] According to a method for optimizing the storage scheduling of railway prefabricated beam yards based on reinforcement learning provided by the present invention, the simulation operation logic construction of the business logic model is performed to obtain a simulation model of the target beam yard, including: Based on the business logic model, the business module of the target beam yard is constructed using an object-oriented method; wherein the business module includes a production module, a storage module and an erection module; Based on the behaviors and interaction modes of the modules of the target beam yard, the operation behavior parameters of the target beam yard are determined and displayed to establish a simulation model of the target beam yard.

[0009] According to a reinforcement learning-based railway prefabricated beam yard storage scheduling optimization method provided by the present invention, the target beam yard also includes a beam moving machine and a bridge erecting machine; After determining and displaying the operation behavior parameters of the target beam field based on the behaviors and interaction modes of the modules of the target beam field to establish a simulation model of the target beam field, the method further includes: Based on the beam yard shape of the target beam yard, the motion rule modeling is performed on the beam moving machine, and the motion rule modeling is performed on the bridge erecting machine.

[0010] According to a reinforcement learning-based optimization method for scheduling railway prefabricated beams, the objective function includes: beam moving machine movement distance, beam yard utilization rate, and labor utilization rate; The method of optimizing the operation behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters, comprises: Based on the movement distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operation behavior parameters.

[0011] According to a method for optimizing the storage schedule of railway prefabricated beam yards based on reinforcement learning provided by the present invention, the operation behavior parameters are optimized based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters, comprising: Using the predetermined objective function as a main evaluation parameter and using the operation behavior parameters of the target beam field as auxiliary evaluation parameters; The main evaluation parameters and the auxiliary evaluation parameters are input into a deep neural network based on reinforcement learning to obtain optimized parameters, so as to optimize the beam storage schedule of the target beam yard based on the optimized parameters.

[0012] The present invention also provides a device for optimizing the scheduling of railway prefabricated beams in a beam storage yard based on reinforcement learning, comprising the following units: An acquisition unit is used to acquire a pre-established simulation model of a target beam field; wherein the simulation model includes operation behavior parameters of the target beam field, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam field; The optimization unit is used to optimize the operation behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the above-described methods for optimizing the storage scheduling of railway prefabricated beam yards based on reinforcement learning.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for optimizing the storage scheduling of railway prefabricated beam yards based on reinforcement learning as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for optimizing the scheduling of railway prefabricated beams in a yard based on reinforcement learning.

[0016] The present invention provides a method and device for optimizing the storage and scheduling of railway prefabricated beam yards based on reinforcement learning, by obtaining a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; based on a predetermined objective function, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the storage and scheduling of beams of the target beam yard based on the optimized operation behavior parameters. It can be seen that the present invention establishes a corresponding simulation model, the model parameters are easy to adjust, and the state of the simulation process can be interfered with at any time during operation, and it can interact more closely with the working conditions of the beam yard; based on a predetermined objective function, the model behavior is adjusted using a deep neural network based on reinforcement learning, breaking through the optimization technology of storage and scheduling of beams under the constraints of multiple resources such as pedestals, equipment, and personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the method for optimizing the storage scheduling of railway prefabricated beam yard based on reinforcement learning provided by the present invention.

[0019] Figure 2 It is a schematic diagram of spatial plane modeling of a beam yard provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the overall framework of the reinforcement learning provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of a railway prefabricated beam yard storage scheduling optimization device based on reinforcement learning provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Combine the following Figure 1-Figure 5 The present invention describes a method and device for optimizing the scheduling of railway prefabricated beams based on reinforcement learning.

[0025] Figure 1 is a flow chart of a method for optimizing the scheduling of railway prefabricated beams in a storage yard based on reinforcement learning provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 100, obtaining a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard.

[0026] It should be noted that the railway precast beam yard is a vital facility in railway construction. Its main functions include the manufacture, maintenance, storage and transportation of precast box beams. Through standardized production processes, the precast beam yard can efficiently produce beams that meet design requirements and provide key components for the construction of railway bridges. Storage scheduling is an important part of the production process of the railway precast beam yard, which involves the storage, management and transportation plan of precast beams.

[0027] Specifically, before step 100, the method further includes: Step S100, obtaining a pre-established business logic model of a target beam yard, and performing simulation operation logic construction on the business logic model to obtain a simulation model of the target beam yard; Among them, the business logic model includes a beam manufacturing area business logic model and a beam storage area business logic model; the beam manufacturing area business logic model is used to represent the business status of the beam manufacturing area in the target beam yard, and the beam storage area business logic model is used to represent the construction status of the beam storage area in the target beam yard.

[0028] Figure 2 This is a schematic diagram of the beam field space plane modeling provided by the present invention. Figure 2 To be specific, in one embodiment: 1. Establish a business logic model for the beam manufacturing area, and determine the data model, behavior model and identification scheme for the beam manufacturing area based on the business situation of the beam yard and beam manufacturing area.

[0029] 1.1. Design the morphological characteristics of the beam-making area. The beam-making area is modeled in a matrix structure, consisting of an inner mold, an outer mold, and a vacant position. Each position is aligned with the pedestal in the beam storage area in the longitudinal direction. The beam-making area is equipped with 3 groups of beam-making units, each of which consists of 2 inner molds and 4 outer molds. The outer mold pedestal outputs the finished product. The spatial modeling of the beam-making area is as follows: Figure 2 shown.

[0030] 1.2. Design the data and production time of the beam making area. The data of each beam in the beam making area includes the beam number and the remaining days for completion; the production sequence of the beams is strictly in accordance with the bridge erection sequence; each unit group can have at most one beam enter the production process every day, and the four outer membrane positions of each unit group are used in turn; the production of the beam takes a total of 4 days, and it can be sent to the beam storage area after completion.

[0031] 2. Establish a business logic model for the beam storage area, and determine the data model, behavior model and identification scheme for the beam storage area based on the construction status of the beam yard and beam storage area.

[0032] 2.1. Design the shape characteristics of the beam storage area. The beam storage area is modeled as a matrix structure. The length and width parameters are determined by the beam field conditions. In this embodiment, there are 10 rows and 10 columns with a total of 100 beam storage pedestals. The spatial modeling of the beam storage area is as follows: Figure 2As shown, at most 2 beams can be stacked on each pedestal, and a total of up to 200 beams can be stored.

[0033] 2.2. Design the data and production time of the beam storage area. The data of each beam in the beam storage area includes the beam number, the remaining days for maintenance, and the remaining days for bridge erection. The beams entering the beam storage area must first undergo a 14-day maintenance process, during which they cannot be moved or stacked. After the maintenance is completed, they must be stored in the beam storage area for a total of 30 days (including the maintenance time) before bridge erection can be carried out.

[0034] 2.3. Design special operation rules for the beam storage area. When the beam yard is full, if new beams continue to be stored, the beam storage area will trigger the beam moving operation, stack the beams that have been cured, and thus free up new beam storage pedestals; when the bridge erection machine turns around, it is required to clear all beams in a specific rectangular area in the beam storage area, and this operation needs to be planned in advance; when the beam yard is full and cannot be stacked, the beam manufacturing area can be temporarily ordered to suspend production for one day to relieve the pressure on the beam storage area.

[0035] Furthermore, in step S100, the business logic model is simulated and run logically to obtain a simulation model of the target beam yard, including: Step S110: Based on the business logic model, use an object-oriented method to construct a business module of the target beam yard; wherein the business module includes a production module, a storage module and an erection module.

[0036] Step S120: Based on the behaviors and interaction modes of the modules of the target beam yard, determining and displaying the operation behavior parameters of the target beam yard to establish a simulation model of the target beam yard.

[0037] The following is a specific description in conjunction with a specific embodiment. Based on the above embodiment, in one embodiment: 3. Construction of simulation operation logic. Adopting object-oriented method, construct beam yard production, storage, and erection business modules respectively, design the behavior and interaction mode of each module according to beam yard business, and count and display the important parameters of the simulation process (i.e. the above-mentioned operation behavior parameters).

[0038] 3.1. Behavior and interface of beam making module. The beam making module is responsible for checking the beam making plan, adding the beams to be produced to the production area according to the plan, monitoring the production of beams, counting the completed beams, and applying for beam storage to the beam storage module; receiving the stop signal from the beam storage area, and stopping for one day each time a signal is received.

[0039] 3.2. Behavior and interface of the beam storage module. The beam storage module is responsible for monitoring and managing the maintenance and other timing operations of beams in the beam storage area; receiving the beam storage application from the beam manufacturing module, and processing the beam storage operation, and moving and stacking beams when necessary; sending a stoppage command to the beam manufacturing module when the beam storage area is close to full to prevent the beam yard from being overfilled; receiving the beam erection application from the beam erection module, and being responsible for finding the beams to be erected and moving them to the upper bridge area (such as Figure 2 The row of beam storage area closest to the railway bridge); receiving the turning request of the beam erection module and being responsible for clearing the designated area required for the bridge erection machine to turn around.

[0040] 3.3 Behavior and interface of the beam erection module. The beam erection module is responsible for detecting the beam erection plan and applying to the beam storage module for beam erection according to the plan; monitoring the mileage and direction of beam erection, controlling the speed of bridge erection, and applying to the beam storage area for the bridge erection machine to turn around in a timely manner.

[0041] 3.4. Control module behavior. The system calls related modules in the order of beam manufacturing, beam storage, and beam erection in a daily cycle, manages various public parameters in the simulation process, and counts and displays the status of the beam yard and key simulation parameters.

[0042] Furthermore, the target beam yard also includes a beam moving machine and a bridge erecting machine.

[0043] After step S120, the method further includes: Based on the beam yard shape of the target beam yard, the motion rule modeling is performed on the beam moving machine, and the motion rule modeling is performed on the bridge erecting machine.

[0044] The following is a specific description in conjunction with a specific embodiment. Based on the above embodiment, in one embodiment: 4. Modeling of the movement rules of the beam moving machine. All the movement of beams in the beam yard needs to be completed by the beam moving machine. According to the shape of the beam yard, the movement of the beam moving machine has its specific specifications.

[0045] 4.1. Beam moving machine movement specifications. There is usually a transverse movement area between the beam manufacturing area and the beam storage area of ​​the beam yard (sometimes between the beam storage area and the railway bridge). Any transverse movement of the beam moving machine must be carried out in the transverse movement area, but there is no restriction on longitudinal movement. Figure 2 The circled numbers in the middle mark four typical movement routes.

[0046] 4.2. Beam moving machine loss specification. The movement of the beam moving machine will generate a lot of fuel consumption, which is an important optimization indicator in the field of beam field optimization. Due to its heavy weight, the loss of the beam moving machine when moving a specific distance without load is equivalent to 70% of the loss of the beam lifting movement.

[0047] 5. Modeling of the movement rules of the bridge erection machine. The bridge erection operation is completed by the bridge erection machine, which has its own unique movement and U-turn mode, which has a great impact on the beam yard and needs to be modeled and described.

[0048] 5.1. Rules for turning the bridge erector. The beam yard is usually built in the middle of the line. When erecting a bridge, it will first proceed in one direction. After completion, the bridge erector will turn around and proceed in the other direction until all the projects are completed. The bridge erector can only turn around in the beam yard. Due to its large length, all beams in a specific area of ​​the beam storage area must be cleared when turning around ( Figure 2 The part enclosed by the 7*3 dotted frame in the middle of the beam storage area).

[0049] 5.2. Bridge erection speed rules. The number of beams that a bridge erection machine can erect every day depends on the distance between the erection location and the beam yard. Within 8 km, 3 beams per day, and 8-15 km, 2 beams per day. Considering the optimal storage of beams in the beam yard, the bridge erection operation is carried out about 30 days later than the beam manufacturing operation.

[0050] Step 200: Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0051] It should be noted that the objective function includes: the moving distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor.

[0052] Specifically, step 200 includes: Based on the movement distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operation behavior parameters.

[0053] Figure 3 This is a schematic diagram of the overall framework of the reinforcement learning provided by the present invention. Figure 3 To provide a specific description, based on the above embodiments, in one embodiment: 6. Establish a multi-objective optimization plan. Take the total movement distance of the beam moving machine, the utilization rate of the beam field, and the utilization rate of labor as the objective functions, optimize and adjust the operating behavior parameters of each module in the simulation process, and form a multi-objective optimization beam storage scheduling plan. Figure 3 The overall framework of reinforcement learning is presented.

[0054] 6.1. Beam moving machine movement distance. During the simulation process, the distance of each beam moving includes the distance d1 of the beam moving machine moving from the current position to the beam starting position without load and the distance d2 of the beam moving machine moving from the beam starting position to the beam target position when the beam is lifted. The sum of the two is the moving distance of this beam moving. The sum of the beam moving distances of all operations in the entire life cycle of the beam field is counted as one of the objective functions.

[0055] 6.2. Utilization rate of beam yard. If the construction area of ​​the beam yard is too large, it will lead to waste of resources. If it is too small, it will lead to too frequent stacking of beams. In order to make full use of the site area, the ratio between the number of beams stored in the beam yard every day and the maximum number of beams stored is counted and accumulated during the simulation process as one of the objective functions.

[0056] 6.3. Labor utilization rate. The beam manufacturing area sometimes suspends or slows down its work because the beam storage area is full, but the various expenses related to the workers will not be suspended. During the simulation process, the cumulative number of downtime days in the beam manufacturing area during the entire life cycle is counted as one of the objective functions to measure the work efficiency of the beam yard.

[0057] Specifically, step 200 also includes: Step 210: Use the predetermined objective function as a main evaluation parameter, and use the operation behavior parameters of the target beam field as auxiliary evaluation parameters.

[0058] Step 220: Input the main evaluation parameters and the auxiliary evaluation parameters into a deep neural network based on reinforcement learning to obtain optimized parameters, so as to optimize the storage schedule of the target beam yard based on the optimized parameters.

[0059] The following is a specific description in conjunction with a specific embodiment. Based on the above embodiment, in one embodiment: 7. Neural network model construction. A deep neural network based on reinforcement learning is used as the optimization algorithm to build a reinforcement learning framework covering four links: simulation engine, neural network, preference parameters, and optimization parameters.

[0060] 7.1. Preference parameters. Preference parameters are the input of the simulation engine and the output of the neural network. In this embodiment, the neural network does not directly control the operation of a specific beam, but uses the preference parameters as an intermediary to adjust the behavior tendency of the beam storage module; preference parameters control various selection tendencies in the beam storage and beam transfer processes. There are 10 preference parameters in total, which can be found in Figure 3 Preferences section.

[0061] 7.2, Evaluation parameters. The evaluation parameters are the output of the simulation engine and the input of the neural network. The neural network determines whether the current preference parameter settings need to be adjusted based on the evaluation parameters; the evaluation parameters include the three main evaluation parameters in step 6 and the four auxiliary evaluation parameters obtained by statistics during the simulation process, which can be seen in Figure 3 Evaluation parameters section.

[0062] 7.3. Neural network model design. The neural network in this embodiment has a total of 4 layers, including 7 neurons in the input layer, 128 neurons in the middle layer, 64 neurons in the middle second layer, and 10 neurons in the output layer. The neurons are connected using the tansig activation function, such as Figure 3The neural network part is shown below.

[0063] The above is a step description of the method for optimizing the scheduling of railway prefabricated beam yard based on reinforcement learning provided by the present invention. From the description of the above steps, it can be seen that according to the method for optimizing the scheduling of railway prefabricated beam yard based on reinforcement learning provided by the present invention, a simulation model of the target beam yard established in advance is obtained; wherein the simulation model includes the operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; based on a predetermined objective function, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the scheduling of the target beam yard based on the optimized operation behavior parameters. It can be seen that the present invention establishes a corresponding simulation model, the model parameters are easy to adjust, and the state of the simulation process can be interfered at any time during operation, and it can interact more closely with the working conditions of the beam yard; based on a predetermined objective function, the model behavior is adjusted using a deep neural network based on reinforcement learning, breaking through the optimization technology of scheduling the scheduling of beams under the constraints of multiple resources such as pedestals, equipment, and personnel.

[0064] The following is a description of the railway prefabricated beam yard inventory scheduling optimization device based on reinforcement learning provided by the present invention. The railway prefabricated beam yard inventory scheduling optimization device based on reinforcement learning described below and the railway prefabricated beam yard inventory scheduling optimization method based on reinforcement learning described above can be referenced to each other.

[0065] Figure 4 is a structural schematic diagram of a railway prefabricated beam yard storage scheduling optimization device based on reinforcement learning provided by the present invention, such as Figure 4 As shown, the railway prefabricated beam yard storage scheduling optimization device based on reinforcement learning provided by the present invention comprises: An acquisition unit 401 is used to acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; The optimization unit 402 is used to optimize the operating behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0066] The device for optimizing the storage and scheduling of railway prefabricated beam yards based on reinforcement learning provided by the present invention obtains a simulation model of a pre-established target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; based on a predetermined objective function, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the storage and scheduling of beams of the target beam yard based on the optimized operation behavior parameters. It can be seen that the present invention establishes a corresponding simulation model, the model parameters are easy to adjust, and the state of the simulation process can be interfered with at any time during operation, and it can interact more closely with the working conditions of the beam yard; based on a predetermined objective function, the model behavior is adjusted using a deep neural network based on reinforcement learning, breaking through the optimization technology of storage and scheduling of beams under the constraints of multiple resources such as pedestals, equipment, and personnel.

[0067] Based on the above embodiment, in this embodiment, the device further includes a simulation unit, which is specifically used for: Before obtaining the simulation model of the pre-established target beam yard, obtaining the pre-established business logic model of the target beam yard, and performing simulation operation logic construction on the business logic model to obtain the simulation model of the target beam yard; Among them, the business logic model includes a beam manufacturing area business logic model and a beam storage area business logic model; the beam manufacturing area business logic model is used to represent the business status of the beam manufacturing area in the target beam yard, and the beam storage area business logic model is used to represent the construction status of the beam storage area in the target beam yard.

[0068] Based on the above embodiments, in this embodiment, the simulation unit is specifically used for: Based on the business logic model, the business module of the target beam yard is constructed using an object-oriented method; wherein the business module includes a production module, a storage module and an erection module; Based on the behaviors and interaction modes of the modules of the target beam yard, the operation behavior parameters of the target beam yard are determined and displayed to establish a simulation model of the target beam yard.

[0069] Based on the above embodiment, in this embodiment, the target beam yard further includes a beam moving machine and a bridge erecting machine; The device also includes a modeling unit, which is specifically used for: Based on the behavior and interaction mode of each module of the target beam yard, the operation behavior parameters of the target beam yard are determined and displayed to establish a simulation model of the target beam yard. Then, based on the beam yard morphology of the target beam yard, the motion rules of the beam moving machine are modeled, and the motion rules of the bridge erecting machine are modeled.

[0070] Based on the above embodiment, in this embodiment, the objective function includes: the movement distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor; The optimization unit 402 is specifically used for: Based on the movement distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operation behavior parameters.

[0071] Based on the above embodiment, in this embodiment, the optimization unit 402 is specifically used for: Using the predetermined objective function as a main evaluation parameter and using the operation behavior parameters of the target beam field as auxiliary evaluation parameters; The main evaluation parameters and the auxiliary evaluation parameters are input into a deep neural network based on reinforcement learning to obtain optimized parameters, so as to optimize the storage schedule of the target beam yard based on the optimized parameters.

[0072] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the railway prefabricated beam yard storage scheduling optimization method based on reinforcement learning, and the method includes: Acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0073] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the railway prefabricated beam yard storage scheduling optimization method based on reinforcement learning provided by the above methods, the method comprising: Acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0075] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the railway prefabricated beam yard storage scheduling optimization method based on reinforcement learning provided by the above methods, the method comprising: Acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operating behavior parameters.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing the scheduling of railway prefabricated beams based on reinforcement learning, characterized in that: include: Acquire a pre-established simulation model of a target beam yard; wherein the simulation model includes operation behavior parameters of the target beam yard, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam yard; Based on a predetermined objective function, the operating behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operating behavior parameters.

2. The method for optimizing the storage schedule of railway prefabricated beam yard based on reinforcement learning according to claim 1 is characterized in that: Before obtaining the pre-established simulation model of the target beam field, the method further includes: Acquire a pre-established business logic model of a target beam yard, and perform simulation operation logic construction on the business logic model to obtain a simulation model of the target beam yard; Among them, the business logic model includes a beam manufacturing area business logic model and a beam storage area business logic model; the beam manufacturing area business logic model is used to represent the business status of the beam manufacturing area in the target beam yard, and the beam storage area business logic model is used to represent the construction status of the beam storage area in the target beam yard.

3. The method for optimizing the storage schedule of railway prefabricated beam yard based on reinforcement learning according to claim 2 is characterized in that: The step of constructing a simulation operation logic of the business logic model to obtain a simulation model of the target beam yard includes: Based on the business logic model, the business module of the target beam yard is constructed using an object-oriented method; wherein the business module includes a production module, a storage module and an erection module; Based on the behaviors and interaction modes of the modules of the target beam yard, the operation behavior parameters of the target beam yard are determined and displayed to establish a simulation model of the target beam yard.

4. The method for optimizing the storage schedule of railway prefabricated beam yard based on reinforcement learning according to claim 3 is characterized in that: The target beam yard also includes a beam moving machine and a bridge erecting machine; After determining and displaying the operation behavior parameters of the target beam field based on the behaviors and interaction modes of the modules of the target beam field to establish a simulation model of the target beam field, the method further includes: Based on the beam yard shape of the target beam yard, the motion rule modeling is performed on the beam moving machine, and the motion rule modeling is performed on the bridge erecting machine.

5. The method for optimizing the storage schedule of railway prefabricated beam yard based on reinforcement learning according to claim 1, characterized in that: The objective function includes: the moving distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor; The method of optimizing the operation behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters, comprises: Based on the movement distance of the beam moving machine, the utilization rate of the beam yard and the utilization rate of labor, the operation behavior parameters are optimized using a deep neural network based on reinforcement learning, so as to optimize the beam storage schedule of the target beam yard based on the optimized operation behavior parameters.

6. The method for optimizing the storage schedule of railway prefabricated beam yard based on reinforcement learning according to claim 1, characterized in that: The method of optimizing the operation behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters, comprises: Using the predetermined objective function as a main evaluation parameter and using the operation behavior parameters of the target beam field as auxiliary evaluation parameters; The main evaluation parameters and the auxiliary evaluation parameters are input into a deep neural network based on reinforcement learning to obtain optimized parameters, so as to optimize the beam storage schedule of the target beam yard based on the optimized parameters.

7. A railway prefabricated beam yard storage scheduling optimization device based on reinforcement learning, characterized in that: include: An acquisition unit is used to acquire a pre-established simulation model of a target beam field; wherein the simulation model includes operation behavior parameters of the target beam field, and the operation behavior parameters are used to represent the behavior and interaction mode of each module of the target beam field; The optimization unit is used to optimize the operation behavior parameters based on a predetermined objective function using a deep neural network based on reinforcement learning, so as to optimize the storage schedule of the target beam yard based on the optimized operation behavior parameters.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for optimizing the storage scheduling of railway prefabricated beam yard based on reinforcement learning as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for optimizing the storage scheduling of railway prefabricated beam yard based on reinforcement learning as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the method for optimizing the storage scheduling of railway prefabricated beam yard based on reinforcement learning as described in any one of claims 1 to 6.

Images