Automatic guided vehicle control method and device, automatic guided vehicle and control system thereof

Abstract

The application discloses an automatic guided vehicle control method and device, an automatic guided vehicle and a control system thereof. The method comprises the following steps: acquiring delay influence factor state values corresponding to delay influence factors of each automatic guided vehicle, wherein the delay influence factors are factors causing the actual running speed of each automatic guided vehicle to be inconsistent with the planned running speed, and the planned running speed is a speed planned in advance for each automatic guided vehicle by a scheduling system of each automatic guided vehicle; inputting the delay influence factor state values into a delay step prediction model to process the delay influence factor state values by using the delay step prediction model, so as to obtain predicted delay steps of each automatic guided vehicle; and determining a control strategy of each automatic guided vehicle according to the predicted delay steps. The application solves the technical problem that the delay of part of AGVs in the related art causes the scheduling plan of the AGV system to be disordered, the reliability of the AGV scheduling system is poor, and there is a safety hazard.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, in particular to an automatic guided vehicle control method and device, an automatic guided vehicle and a control system thereof. BACKGROUND

[0002] An automatic guided vehicle (AGV) automatic warehouse system usually includes a task allocation module and a path planning module. The path planning module is used to cooperatively plan the execution task paths of all AGVs to avoid conflicts and efficiently complete tasks. In an AGV scheduling system, a path planning algorithm usually assumes that all AGVs have the same running speed to plan a conflict-free path, and does not consider the speed differences caused by complex factors of each AGV in the actual application scenario. In the actual application scenario, due to factors such as load and control differences, the AGVs run with delays in the path, so that the actual running speed of the AGVs is not completely the same as the speed planned by the system, causing the AGVs to travel out of sync with each other. The delay of some AGVs will cause the scheduling plan to be disordered, resulting in poor reliability and inaccurate scheduling of the AGV scheduling system, and AGV conflicts may occur, causing economic losses and safety hazards.

[0003] At present, no effective solution has been proposed to solve the above problems. SUMMARY

[0004] Embodiments of the present application provide an automatic guided vehicle control method and device, an automatic guided vehicle and a control system thereof, to at least solve the technical problem that the delay of some AGVs in the related art causes the scheduling plan of the AGV system to be disordered, resulting in poor reliability of the AGV scheduling system and safety hazards.

[0005] According to an aspect of some embodiments of the present application, a method for controlling an automated guided vehicle is provided. The method comprises: obtaining a delay factor state value corresponding to each delay factor of each automated guided vehicle, wherein the delay factor is a factor causing a difference between an actual running speed and a planned running speed of each automated guided vehicle, the planned running speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay factor state value is a numerical value corresponding to each delay factor; inputting the delay factor state value into a delay step prediction model to process the delay factor state value by using the delay step prediction model, to obtain a predicted delay step of each automated guided vehicle, wherein the predicted delay step is used to evaluate a speed difference between the actual running speed and the planned running speed of each automated guided vehicle, and the delay step prediction model is a model trained by reinforcement learning using a plurality of training data; and determining a control strategy of each automated guided vehicle according to the predicted delay step.

[0006] Optionally, obtaining the delay factor state value corresponding to each delay factor of each automated guided vehicle comprises: determining each delay factor; obtaining current state information of each automated guided vehicle; and determining the delay factor state value corresponding to each delay factor of each automated guided vehicle according to the current state information.

[0007] Optionally, before inputting the delay factor state value into the delay step prediction model to process the delay factor state value by using the delay step prediction model to obtain the predicted delay step of each automated guided vehicle, the method further comprises: obtaining the plurality of training data comprising a plurality of sample delay factor state values and a plurality of sample predicted delay steps corresponding to the plurality of sample delay factor state values; and training the plurality of training data to obtain the delay step prediction model.

[0008] Optionally, training the plurality of training data to obtain the delay step prediction model comprises: obtaining a weight value of each of the plurality of sample delay factor state values; generating state data of the delay step prediction model according to the weight value of each of the plurality of sample delay factor state values and the plurality of sample delay factor state values; generating action data of the delay step prediction model according to each of the plurality of sample delay factor state values; determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; and training the delay step prediction model according to the state data, the action data, and the reward value.

[0009] Optionally, the weight value of each of the plurality of sample delay factor state values is obtained by determining the delay impact degree of the sample delay factor corresponding to the plurality of sample delay factor state values on each of the AGVs, and determining the weight value according to the delay impact degree.

[0010] Optionally, the state data of the delay step number prediction model is generated according to the weight value of each of the plurality of sample delay factor state values and the plurality of sample delay factor state values, including generating a sample state matrix according to the plurality of sample delay factor state values, generating a weight matrix according to the weight value, and determining the state data according to the Hadamard product of the state matrix and the weight matrix.

[0011] Optionally, the action data of the delay step number prediction model is generated according to each of the plurality of sample delay factor state values, including determining the AGV with the fastest average speed in a predetermined time period as a reference AGV from the plurality of sample delay factor state values, obtaining the delay step number of each AGV other than the reference AGV in a delay step number prediction period, and determining the delay step number between each AGV as the action data.

[0012] Optionally, the reward value is determined according to the action data, including determining the reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay step number of the i-th AGV, A i represents the predicted delay step number of the i-th AGV, and n represents the total number of AGVs.

[0013] Optionally, the delay step number prediction model is trained according to the state data, the action data and the reward value, including adjusting the weight value by a weight value adjustment formula according to the reward value to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the j-th sample delay factor state value of the i-th AGV after adjustment, represents the initial weight value of the j-th sample delay factor state value of the i-th AGV, R i represents the reward value of the i-th AGV, R k represents the reward value of the k-th AGV; and the delay step number prediction model is trained according to the state data, the action data and the adjusted reward value.

[0014] Optionally, the training of the delay step prediction model according to the state data, the action data and the reward value comprises: adjusting the weight value according to the reward value to adjust a weight parameter of the delay step prediction model; inputting the state data into the delay step prediction model to obtain output data of the delay step prediction model; and stopping the training of the delay step prediction model when a difference between the predicted action data corresponding to the output data and the action data is less than a preset difference value.

[0015] Optionally, the control strategy of each automatic guided vehicle is determined according to the predicted delay step, and the control strategy comprises: when the predicted delay step is less than a delay step threshold, determining that the control strategy is to control each automatic guided vehicle to run in a current running mode; and when the predicted delay step is not less than the delay step threshold, determining that the control strategy is to adjust a running mode of all or part of the automatic guided vehicles.

[0016] According to another aspect of the embodiment of the present application, an automatic guided vehicle control device is further provided, comprising: a first acquisition unit configured to acquire a delay factor state value corresponding to each delay factor of each automatic guided vehicle, wherein the delay factor is a factor causing an actual running speed of each automatic guided vehicle to be inconsistent with a planned running speed of each automatic guided vehicle, the planned running speed is a speed pre-planned by a scheduling system of each automatic guided vehicle, and the delay factor state value is a numerical value corresponding to each delay factor; a processing unit configured to input the delay factor state value into a delay step prediction model to process the delay factor state value by using the delay step prediction model to obtain a predicted delay step of each automatic guided vehicle, wherein the predicted delay step is used to evaluate a speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, and the delay step prediction model is a model trained by using a plurality of sets of training data through reinforcement learning; and a determination unit configured to determine a control strategy of each automatic guided vehicle according to the predicted delay step.

[0017] Optionally, the first acquisition unit comprises: a first determination sub-unit configured to determine each delay factor; a first acquisition sub-unit configured to acquire current state information of each automatic guided vehicle; and a second determination sub-unit configured to determine the delay factor state value corresponding to each delay factor of each automatic guided vehicle according to the current state information.

[0018] Optionally, the AGV control device further comprises: a second acquisition unit, configured to acquire the plurality of sets of training data including a plurality of sample delay factor state values and a plurality of sample predicted delay steps corresponding to the plurality of sample delay factor state values before inputting the delay factor state value into the delay step prediction model and processing the delay factor state value by using the delay step prediction model to obtain the predicted delay step of each AGV.

[0019] Optionally, the training unit comprises: a second acquisition subunit, configured to acquire a weight value of each of the plurality of sample delay factor state values; a first generation subunit, configured to generate state data of the delay step prediction model according to the weight value of each of the plurality of sample delay factor state values and the plurality of sample delay factor state values; a second generation subunit, configured to generate action data of the delay step prediction model according to each of the plurality of sample delay factor state values; a third determination subunit, configured to determine a reward value according to the action data, wherein the reward value is used to optimize the weight value; and a training subunit, configured to train the delay step prediction model according to the state data, the action data and the reward value.

[0020] Optionally, the second acquisition subunit comprises: a first determination module, configured to determine a delay influence degree of a sample delay factor corresponding to the plurality of sample delay factor state values on each AGV; and a second determination module, configured to determine the weight value according to the delay influence degree.

[0021] Optionally, the first generation subunit comprises: a first generation module, configured to generate a sample state matrix according to the plurality of sample delay factor state values; a second generation module, configured to generate a weight matrix according to the weight value; and a third determination module, configured to determine the state data according to a Hadamard product of the state matrix and the weight matrix.

[0022] Optionally, the second generation subunit comprises: a fourth determination module, configured to determine that an AGV with the fastest average speed in a predetermined time period among the plurality of sample delay factor state values is a reference AGV; a first acquisition module, configured to acquire a delay step of each AGV other than the reference AGV in a delay step prediction period; and a fifth determination module, configured to determine the delay step between each AGV as the action data.

[0023] Optionally, the third determining subunit includes: a sixth determining module, configured to determine the reward value through a reward function, wherein the reward function is: R represents the reward value. A represents the actual delay step number of the i-th automated guided vehicle. i This represents the predicted delay steps for the i-th automated guided vehicle, and n represents the total number of automated guided vehicles.

[0024] Optionally, the training subunit includes: a first adjustment module, configured to adjust the weight value using the reward value through a weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: w ij The weight value represents the adjusted state value of the j-th sample delay influencing factor of the i-th automated guided vehicle. R represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automated guided vehicle. i R represents the reward value for the i-th automated guided vehicle. k The reward value represents the reward value of the kth automated guided vehicle; the training module is used to train a delay step prediction model based on the state data, the action data, and the adjusted reward value.

[0025] Optionally, the training subunit includes: a first adjustment module, configured to adjust the weight value according to the reward value, so as to adjust the weight parameters of the delayed step prediction model; a second acquisition module, configured to input the state data into the delayed step prediction model to obtain the output data of the delayed step prediction model; and a stop module, configured to stop the training operation of the delayed step prediction model when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference.

[0026] Optionally, the determining unit includes: a fourth determining subunit, configured to determine the control strategy as follows when the predicted delay step number is less than the delay step number threshold: control each of the automated guided vehicles to operate according to the current operating mode; and a fifth determining subunit, configured to determine the control strategy as follows when the predicted delay step number is not less than the delay step number threshold: adjust all or part of the operating modes of each of the automated guided vehicles.

[0027] According to another aspect of the present invention, an automated guided vehicle is also provided, wherein the automated guided vehicle uses any of the above-described automated guided vehicle control methods.

[0028] According to another aspect of the embodiments of the present application, there is also provided an automated guided vehicle control system using the automated guided vehicle control method of any one of the above.

[0029] According to another aspect of the embodiments of the present application, there is also provided a computer readable storage medium comprising a stored program, wherein the program performs the automated guided vehicle control method of any one of the above.

[0030] According to another aspect of the embodiments of the present application, there is also provided a processor configured to execute a program, wherein the program performs the automated guided vehicle control method of any one of the above.

[0031] According to another aspect of the embodiments of the present application, there is also provided a computer program product comprising computer instructions configured to perform the automated guided vehicle control method of any one of the above when executed by a processor.

[0032] In the embodiments of the present application, the delay influencing factor state values corresponding to the delay influencing factors of each automated guided vehicle are obtained, wherein the delay influencing factor is a factor causing the actual running speed of each automated guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay influencing factor state value is a value corresponding to each delay influencing factor; the delay influencing factor state values are input into a delay step number prediction model to process the delay influencing factor state values by using the delay step number prediction model, to obtain the predicted delay step number of each automated guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each automated guided vehicle, the delay step number prediction model is a model trained by machine learning using a plurality of sets of training data, and each set of the plurality of sets of training data comprises a sample delay influencing factor state value and a sample predicted delay step number corresponding to the sample delay influencing factor state value; and the control strategy of each automated guided vehicle is determined according to the predicted delay step number. Through the above technical solutions provided by the present application, the delay step number prediction model is pre-trained, so that after the delay influencing factor state values of each AGV are obtained, the delay step number prediction model is used to determine the predicted delay step number of each AGV, and the predicted delay step number is used as a factor for controlling the running of each AGV, thereby accurately predicting the running step number of the AGV, obtaining the relative delay between the AGVs, and achieving the technical effect that the actual AGV and the theoretically planned path are consistent in the actual application scenario, realizing accurate prediction of the AGV scheduling delay, and thereby solving the technical problems in the related art that the delay of part of the AGVs causes the scheduling plan of the AGV system to be disordered, the reliability of the AGV scheduling system is poor, and there is a safety hazard. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0034] Figure 1 This is a hardware structure block diagram of a mobile terminal for an automated guided vehicle control method according to an embodiment of the present invention.

[0035] Figure 2 This is a flowchart of an automated guided vehicle control method according to an embodiment of the present invention;

[0036] Figure 3 This is a flowchart of an optional automated guided vehicle control method according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of an automated guided vehicle control device according to an embodiment of the present invention. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] As described in the background section, delays in some AGVs in related technologies can cause disruptions in the AGV system's scheduling plan, resulting in poor reliability and potential safety hazards. To address these shortcomings, embodiments of the present invention provide an automated guided vehicle (AGV) control method and apparatus, a computer-readable storage medium, a processor, and a computer program product.

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0042] The methods and embodiments provided in this invention can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an automated guided vehicle (AGV) control method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0043] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the automatic guided vehicle control method in the embodiments of the present application. The processor 102 can execute various functional applications and data processing, i.e., implement the above method, by running the computer program stored in the memory 104. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the mobile terminal through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific examples of the above network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0044] According to the embodiments of the present application, a method embodiment of an automatic guided vehicle control method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0045] Figure 2 is a flowchart of an automatic guided vehicle control method according to the embodiments of the present application, as shown in Figure 2 The automatic guided vehicle control method includes the following steps:

[0046] In step S202, the delay influencing factor state values corresponding to the delay influencing factors of each automatic guided vehicle are obtained, wherein the delay influencing factor is a factor causing the actual running speed of each automatic guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned for each automatic guided vehicle by a dispatching system of each automatic guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor.

[0047] In the embodiment of the present application, when the AGV scheduling system plans the path for each AGV, the AGV in the AGV scheduling system is regarded as the same, and when the AGV is scheduled and planned, the running speed of the AGV set in the theoretically running path is regarded as completely uniform. However, in the actual running process, the AGV will inevitably appear the states of starting acceleration, stable running, deceleration and stopping, and only when it is in stable running, the actual running speed is the same as the theoretically set running speed. Therefore, for the path planning algorithm of multi-AGV collaborative planning, in the actual application scene, there is a conflict in the actual running process although there is a conflict-free path in the theoretical planning, because all AGVs cannot be completely synchronized.

[0048] However, the AGV cannot be completely synchronized due to various factors, such as load, the heavier the load, the slower the speed; and the service life and brand of the vehicle, the acceleration and deceleration of each AGV are different, causing speed difference; in addition, the distance from the target point also triggers different speed control mechanisms of the controller, such as the closer to the target point, the greater the deceleration, to ensure that the AGV stops at the target point in time.

[0049] Here, the load, acceleration, deceleration, current speed and distance from the target point can be selected as the delay prediction parameters, i.e., delay influencing factors.

[0050] It should be noted that the above delay influencing factors can not be limited to this, and different influencing factors can be considered according to the actual situation in different application scenarios.

[0051] Among them, the delay influencing factor state value can be the value corresponding to each delay influencing factor of the AGV in the actual running process, for example, the AGV load value f i , the acceleration f i , the deceleration r i , the current speed v i , and the distance from the target point d i .

[0052] Step S204, input the delay influencing factor state value into the delay step prediction model, to process the delay influencing factor state value by using the delay step prediction model, to obtain the predicted delay step number of each automatic guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, the delay step prediction model is a model trained by machine learning using a plurality of sets of training data, each set of training data in the plurality of sets of training data includes: a sample delay influencing factor state value and a sample predicted delay step number corresponding to the sample delay influencing factor state value, an actual step number corresponding to the sample delay influencing factor state value, a reward function is constructed based on the predicted delay step number and the actual step number, and the model is optimized according to the reward function.

[0053] Here, the state values ​​of delay influencing factors obtained in the above steps can be input into a pre-trained delay step prediction model. The delay step prediction model is then used to process the state values ​​of the delay influencing factors to obtain the predicted delay steps for each AGV, and thus the delay steps for each AGV within the predicted steps can be determined.

[0054] Step S206: Determine the control strategy for each automated guided vehicle based on the predicted delay steps.

[0055] In this embodiment, the operating strategy of each AGV, i.e., the control strategy, can be determined based on the predicted delay steps obtained above. The control strategy can be determined based on the specific predicted delay steps. For example, the control strategy can use the predicted delay steps to determine whether the AGV needs to accelerate (if there are no other vehicles in front of the AGV and no vehicle collision is expected, and the predicted delay steps are high, the AGV can be controlled to accelerate; conversely, if the current speed might cause a collision with the vehicle in front, the AGV's speed can be calculated based on the delay steps to determine that the AGV will neither collide with the vehicle in front nor affect the normal driving of vehicles behind at that speed).

[0056] In this embodiment of the invention, the predicted number of steps is to ensure that the AGVs operate in a coordinated and orderly manner, according to the system settings, neither too fast nor too slow, that is, accurate, so as to prevent scheduling chaos and indirectly prevent collisions.

[0057] Furthermore, after determining the control strategy for each automated guided vehicle (AGV) through the above steps, each AGV can be controlled to operate according to the control strategy to ensure that the AGVs can operate in an orderly manner.

[0058] From the above, in the embodiment of the application, the delay influencing factor state values corresponding to the delay influencing factors of each automated guided vehicle are obtained, wherein the delay influencing factors are factors causing the actual running speed of each automated guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay influencing factor state value is a value corresponding to each delay influencing factor; the delay influencing factor state values are input into a delay step prediction model, so as to process the delay influencing factor state values by using the delay step prediction model, and obtain the predicted delay steps of each automated guided vehicle, wherein the predicted delay steps are used to evaluate the speed difference between the actual running speed and the planned running speed of each automated guided vehicle, and the delay step prediction model is a model obtained by machine learning training using a plurality of sets of training data, each of the plurality of sets of training data including: a sample delay influencing factor state value and a sample predicted delay step corresponding to the sample delay influencing factor state value; and the control strategy of each automated guided vehicle is determined according to the predicted delay steps, which realizes that the delay step prediction model is pre-trained, so that after the delay influencing factor state values of each AGV are obtained, the predicted delay steps of each AGV are determined by using the delay step prediction model, and then the predicted delay steps are used as a factor for controlling the running of each AGV, so that the running steps of the AGV are accurately predicted, the relative delay between the AGVs is obtained, the actual and theoretical planning paths of the AGV in the actual application scenario are kept consistent, and the accurate prediction of the AGV scheduling delay is realized.

[0059] Therefore, by using the above technical solution provided in the embodiment of the application, the technical problem that the delay of part of the AGVs in the related art causes the scheduling plan of the AGV system to be disordered, the reliability of the AGV scheduling system is poor, and there is a safety hazard is solved.

[0060] According to the above embodiment of the application, obtaining the delay influencing factor state values corresponding to the delay influencing factors of each automated guided vehicle can include: determining the delay influencing factors; obtaining the current state information of each automated guided vehicle; and determining the delay influencing factor state values corresponding to the delay influencing factors of each automated guided vehicle according to the current state information.

[0061] In this embodiment, the current state information of each AGV can be obtained, such as load, acceleration, deceleration, current speed, and distance to the target point, so that the delay influencing factor state values of the delay influencing factors can be determined according to the values of these factors.

[0062] According to the above embodiment of the present application, before the delay step number prediction model is used to process the delay affecting factor state value to obtain the predicted delay step number of each AGV, the AGV control method further comprises: obtaining a plurality of sets of training data comprising a sample delay affecting factor state value and a sample predicted delay step number corresponding to the sample delay affecting factor state value, and an actual step number corresponding to the sample delay affecting factor state value; training the plurality of sets of training data to obtain the delay step number prediction model.

[0063] In this embodiment, the historical running data of each AGV in the historical time can be obtained, and the historical running data is analyzed to obtain each set of training data, i.e., the sample delay affecting factor state value and the sample predicted delay step number corresponding to the sample delay affecting factor state value.

[0064] According to the above embodiment of the present application, the plurality of sets of training data are trained to obtain the delay step number prediction model, which can comprise: obtaining a weight value of each of the plurality of sample delay affecting factor state values; generating state data of the delay step number prediction model according to the weight value of each of the plurality of sample delay affecting factor state values and the plurality of sample delay affecting factor state values; generating action data of the delay step number prediction model according to each of the plurality of sample delay affecting factor state values; determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; and training the delay step number prediction model according to the state data, the action data, and the reward value.

[0065] In this embodiment, the weight value of each of the plurality of sample delay affecting factor state values is obtained, and the weight value represents the influence degree of each sample delay affecting factor state value on the delay of each AGV.

[0066] In addition, in the embodiment of the present application, the core elements of reinforcement learning are state, action, and reward function, which are the key to modeling. The above core elements will be described in detail below, and will not be described here.

[0067] In the above embodiment, obtaining the weight value of each of the plurality of sample delay affecting factor state values comprises: determining the delay affecting degree of a sample delay affecting factor corresponding to the plurality of sample delay affecting factor state values on each AGV; and determining the weight value according to the delay affecting degree.

[0068] In actual AGV scheduling, the influence effects of each affecting factor on the delay of AGV running are different, so the factor weight value is used to determine the influence degree of the state factor of each AGV on the delay of AGV running.

[0069] In the above embodiments, generating state data for the delay step prediction model based on the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values ​​includes: generating a sample state matrix based on the multiple sample delay influencing factor state values; generating a weight matrix based on the weight values; and determining the state data based on the Hadamard product of the state matrix and the weight matrix.

[0070] In this embodiment, a sample state matrix can be generated based on the state values ​​of multiple sample delay influencing factors, such as: State: AGV load value f i acceleration a i deceleration r i Current speed v i Distance d from the target point i Assuming we consider three AGVs, the state matrix can be represented as:

[0071] In actual AGV scheduling, the effects of various influencing factors on AGV running delay are different. Therefore, a weight matrix is ​​introduced here to set the degree of influence of the state factors of each AGV on the AGV running delay. Each element of the weight matrix corresponds to the weight of each state parameter.

[0072] Next, based on the state matrix and weight matrix, the model input can be set as the Hadamard product of the state matrix S and the weight matrix W, i.e., the element-wise multiplication of the two matrices. Therefore, the model input is represented as:

[0073] Each element of the W matrix can be initialized to 1 (by default, all influencing factors of all AGVs have the same weight), or set to a specific value according to the characteristics of each AGV, and then dynamically adjusted according to the reward function during the model optimization process.

[0074] According to the above embodiments of the present invention, the action data for generating a delay step prediction model based on each of the state values ​​of multiple sample delay influencing factors includes: determining the automated guided vehicle (AGV) with the fastest average speed within a predetermined time period corresponding to the state values ​​of multiple sample delay influencing factors as the reference AGV; obtaining the delay steps of other vehicles in each AGV except the reference AGV within the delay step prediction period; and determining the delay steps between each AGV as action data.

[0075] In this embodiment, motion data can be set based on the number of steps the AGV takes. Since the synchronization between AGVs is relative—that is, if we set the AGV with the fastest average speed over a period of time as the standard—the operating state of other AGVs can be represented as a delay relative to the fastest AGV. Therefore, we set the motion to the relative delay steps between AGVs.

[0076] Assuming five steps as a prediction cycle, a maximum of four steps can be delayed. 0 represents the fastest AGV travel speed, a delay of 0, -1 represents a relative fastest AGV delay of 1 step, -2 represents a relative fastest AGV delay of 2 steps, and so on, a total of action selection. Among them, indicates that one of the three AGVs is the fastest, that is, the action is 0; two indicates that for the remaining two AGVs, the action of each AGV is selected from the five choices (0, -1, -2, -3, -4). For example, taking three AGVs as an example, the action (-1 0 -3) indicates that AGV2 is the fastest in the future 5 steps, and AGV1 will be relatively delayed by 1 step and AGV3 will be relatively delayed by 3 steps.

[0077] According to the above embodiment of the present application, the reward value is determined according to the action data, comprising: determining the reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay step number of the i-th AGV, A i represents the predicted delay step number of the i-th AGV, and n represents the total number of AGVs.

[0078] Here, the reward function can be set to the inverse of the mean square error of the actual delay step number of the AGV and the predicted delay step number A i : where n represents the number of AGVs. The R value is used as a reward function to update the model training parameters, in addition to which.

[0079] According to the above embodiment of the present application, the delay step number prediction model is trained according to the state data, the action data and the reward value, comprising: adjusting the weight value by a weight value adjustment formula to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents the adjusted weight value of the j-th sample delay influencing factor state value of the i-th AGV, represents the initial weight value of the j-th sample delay influencing factor state value of the i-th AGV, R i represents the reward value of the i-th AGV, R k represents the reward value of the k-th AGV; the delay step number prediction model is trained according to the state data, the action data and the adjusted reward value.

[0080] Since the actual AGVs are not completely the same in practice, that is, the actual delay step number of one or several AGVs may be greatly different from the predicted delay step number, if only the expression form of the reward R is adopted, the prediction effect of the AGVs with the consistent predicted step number and actual step number is not good under the training of the reward function R, therefore, in the embodiment of the present application, the weight of each individual AGV is designed, specifically, the R value corresponding to each AGV is individually taken as the reference value of the weight matrix of the above input, which is expressed as the above weight value adjustment formula. i

[0081] wherein, is the initial value of the weight matrix element (all elements are 1). Based on this expression, the reward and punishment of each AGV can be fed back to the trained network weight parameter, the network can focus on the training, and the convergence speed of the whole model to reach a better strategy can be avoided to a certain extent due to the poor prediction performance of individual AGVs, so that the prediction effect of the AGV is better.

[0082] According to the above embodiment of the present application, the delay step number prediction model is trained according to the state data, the action data and the reward value, comprising: adjusting the weight value according to the reward value to adjust the weight parameter of the delay step number prediction model; inputting the state data into the delay step number prediction model to obtain the output data of the delay step number prediction model; and stopping the training operation of the delay step number prediction model when the difference between the predicted action data corresponding to the output data and the action data is less than the preset difference value.

[0083] In this embodiment, the weight value can be adjusted according to the above reward value, that is, the reward and punishment of each AGV is fed back to the trained network weight parameter, the network can focus on the training, and the convergence speed of the whole model to reach a better strategy can be avoided to a certain extent due to the poor prediction performance of individual AGVs, so that the prediction effect of the AGV is better.

[0084] In the above steps, the AGV state parameter S t at time t is input into the reinforcement learning algorithm to obtain the action parameter A t at time t, the conflict-free path under the set step number is planned according to the action parameter path planning algorithm, and is issued to the AGV.

[0085] When the AGV runs to the set step number (i.e., the delay prediction period, the delay step number prediction is performed every several steps), the delay step number of each AGV is counted, and the reward function R t+1 at time (t+1) is calculated according to the delay step number, and the state parameter S t+1 ​According to a specific reinforcement learning algorithm strategy, the model parameters are updated. The above method can be repeated until the delay prediction mode converges, and the model training is completed.

[0086] According to the above embodiments of the present application, the control strategy of each automatic guided vehicle is determined according to the predicted delay step number, including: when the predicted delay step number is less than the delay step number threshold, determining the control strategy as: controlling each automatic guided vehicle to run according to the current running mode; when the predicted delay step number is not less than the delay step number threshold, determining the control strategy as: adjusting the running mode of all or part of each automatic guided vehicle.

[0087] In this embodiment, the running of each AGV can be controlled according to the size relationship between the predicted delay step number and the delay step number threshold (i.e., the maximum allowable delay step number, within which range the AGV will not collide), so that the AGVs will not collide with each other.

[0088] For example, when the delay step number threshold is 5 steps and the predicted delay step number is 3 steps, the current running speed and running route of each automatic guided vehicle can be controlled without adjustment, i.e., running according to the current running mode; on the contrary, if the predicted delay step number is 8 steps, in this case, it may affect the running of subsequent automatic guided vehicles, so the running mode can be adjusted according to the current running speed and running route of each automatic guided vehicle to determine the orderly running of each automatic guided vehicle.

[0089] Figure 3 is the flowchart of the optional automatic guided vehicle control method according to the embodiments of the present application, as shown in Figure 3 First, the state, action and reward function can be set, then the state is input to obtain the action and control the AGV running; then the real-time delay step number is counted, the reward function is calculated and the state is updated; the model parameters are updated, and when the model converges, the model is saved; in actual application, the model is called to predict the delay step number.

[0090] In addition, in actual application, only the trained model needs to be called for real-time prediction, or the original trained model is continued to be trained based on new data, and the parameter feedback is used to make it more suitable for the application scenario.

[0091] Through the above technical solutions provided by the embodiments of the present application, based on the reinforcement learning technology, by considering various delay influencing factors of the AGV (such as load, acceleration, deceleration, current speed and distance to target point), and dynamically adjusting the factor weight matrix according to the difference between the delay prediction situation and the actual delay situation of each AGV, the convergence of the model is accelerated. Based on the delay situation path planning algorithm, a conflict-free path can be planned more accurately, which improves the running efficiency of the AGV at the source.

[0092] Therefore, by the above technical solutions provided by the embodiments of the present application, the AGV conflict problem caused by the inconsistency between the actual AGV and the theoretical AGV is solved, and the system reliability and the production safety are improved. The reinforcement learning algorithm (such as Double Deep Q Network (DDQN)) is used to predict the relative delay steps between AGVs in a period of time as the basis for AGV path planning, the AGV load value, acceleration, deceleration, current speed and distance to the target point are taken as state parameters, the relative delay steps between AGVs are taken as action parameters, the inverse number of the mean square error of the actual relative delay steps and the predicted relative delay steps of AGVs in this period of time is taken as a reward function to guide the algorithm to train, and the reward function of each AGV is used to feedback the adjustment factor weight matrix according to the difference between AGVs, so as to accurately predict the running steps of AGVs, so as to obtain the relative delay situation between AGVs, so that the actual AGV in the actual application scene is consistent with the theoretical planning path, the AGV conflict problem is solved, the system reliability is improved, and the safety of actual production and transportation is ensured. At the same time, the AGV conflict problem caused by the delay of AGVs out of sync in the actual application scene is also solved, the delay of multiple AGVs is predicted, the path planning algorithm is more accurate in scheduling the actual operation of AGVs, the conflict problem caused by the delay of AGVs is avoided, the running efficiency of the AGV scheduling system is improved, and the safety of actual production is ensured.

[0093] That is, the reinforcement learning algorithm is used to predict the relative delay steps between AGVs in a period of time as the basis for AGV path planning, the AGV load value, acceleration, deceleration, current speed and distance to the target point are taken as state parameters, the relative delay steps between AGVs are taken as action parameters, the inverse number of the mean square error of the actual relative delay steps and the predicted relative delay steps of AGVs in this period of time is taken as a reward function to guide the algorithm to train, and the reward function of each AGV is used to feedback the adjustment factor weight matrix according to the difference between AGVs, so as to accurately predict the running steps of AGVs, so as to obtain the relative delay situation between AGVs, so that the actual AGV in the actual application scene is consistent with the theoretical planning path, and the accurate prediction of AGV scheduling delay is realized.

[0094] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the described action sequence, because according to the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform as necessary, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in each embodiment of the present application.

[0096] According to the embodiments of the present application, an automatic guided vehicle control device for implementing the above automatic guided vehicle control method is further provided, Figure 4 is a schematic diagram of the automatic guided vehicle control device according to the embodiments of the present application, as Figure 4 shown, the device includes a first acquisition unit 401, a processing unit 403, and a determination unit 407. The automatic guided vehicle control device will be described in detail below.

[0097] The first acquisition unit 401 is configured to acquire a delay influencing factor state value corresponding to each delay influencing factor of each automatic guided vehicle, wherein the delay influencing factor is a factor causing the actual running speed of each automatic guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned by a scheduling system of each automatic guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor.

[0098] The processing unit 403 is configured to input the delay influencing factor state value into a delay step prediction model, to process the delay influencing factor state value by using the delay step prediction model, and to obtain a predicted delay step of each automatic guided vehicle, wherein the predicted delay step is used to evaluate the speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, and the delay step prediction model is a model trained by reinforcement learning using a plurality of sets of training data.

[0099] The determination unit 407 is configured to determine a control strategy of each automatic guided vehicle according to the predicted delay step.

[0100] It should be noted that the first acquisition unit 401, the processing unit 403, and the determination unit 407 correspond to steps S202 to S206 in the above embodiments, and the three units have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiments.

[0101] According to the scheme described in the above embodiment of the present application, the first acquisition unit can be used to acquire the delay influencing factor state value corresponding to each delay influencing factor of each automatic guided vehicle, wherein the delay influencing factor is a factor causing the actual running speed of each automatic guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed planned in advance for each automatic guided vehicle by a scheduling system of each automatic guided vehicle, and the delay influencing factor state value is a value corresponding to each delay influencing factor; the processing unit is used to input the delay influencing factor state value into a delay step prediction model, so as to process the delay influencing factor state value by using the delay step prediction model, and obtain the predicted delay step of each automatic guided vehicle, wherein the predicted delay step is used to evaluate the speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, and the delay step prediction model is a model trained by using a plurality of sets of training data through reinforcement learning; and the determination unit is used to determine the control strategy of each automatic guided vehicle according to the predicted delay step, thereby realizing the following technical effects: the delay step prediction model is trained in advance, so that after the delay influencing factor state value of each AGV is acquired, the delay step prediction model is used to determine the predicted delay step of each AGV, and then the predicted delay step is used as a factor for controlling the running of each AGV, so that the running step of the AGV can be accurately predicted, the relative delay between the AGVs can be obtained, and the technical effect that the actual and theoretical planned paths of the AGV in the actual application scenario are consistent is achieved, and the accurate prediction of the AGV scheduling delay is realized.

[0102] Therefore, by using the above technical scheme provided in the embodiments of the present application, the technical problem that the delay of part of the AGVs in the related art causes the scheduling plan of the AGV system to be disordered, the reliability of the AGV scheduling system is poor, and there is a safety hazard is solved.

[0103] Optionally, the first acquisition unit comprises: a first determination subunit configured to determine each delay influencing factor; a first acquisition subunit configured to acquire the current state information of each automatic guided vehicle; and a second determination subunit configured to determine the delay influencing factor state value corresponding to each delay influencing factor of each automatic guided vehicle according to the current state information.

[0104] Optionally, the automatic guided vehicle control device further comprises: a second acquisition unit configured to acquire a plurality of sets of training data comprising a plurality of sample delay influencing factor state values and a plurality of sample predicted delay steps corresponding to the plurality of sample delay influencing factor state values, before the delay influencing factor state value is input into the delay step prediction model, so as to process the delay influencing factor state value by using the delay step prediction model, and obtain the predicted delay step of each automatic guided vehicle; and a training unit configured to train the plurality of sets of training data, and obtain the delay step prediction model.

[0105] Optionally, the training unit comprises: a second obtaining subunit, configured to obtain a weight value of each of the plurality of sample delay factor state values; a first generating subunit, configured to generate state data of the delay step number prediction model according to the weight value of each of the plurality of sample delay factor state values and the plurality of sample delay factor state values; a second generating subunit, configured to generate action data of the delay step number prediction model according to each of the plurality of sample delay factor state values; a third determining subunit, configured to determine a reward value according to the action data, wherein the reward value is used to optimize the weight value; and a training subunit, configured to train the delay step number prediction model according to the state data, the action data and the reward value.

[0106] Optionally, the second obtaining subunit comprises: a first determining module, configured to determine a delay influence degree of a sample delay factor corresponding to the plurality of sample delay factor state values on each of the automated guided vehicles; and a second determining module, configured to determine the weight value according to the delay influence degree.

[0107] Optionally, the first generating subunit comprises: a first generating module, configured to generate a sample state matrix according to the plurality of sample delay factor state values; a second generating module, configured to generate a weight matrix according to the weight value; and a third determining module, configured to determine the state data according to a Hadamard product of the state matrix and the weight matrix.

[0108] Optionally, the second generating subunit comprises: a fourth determining module, configured to determine an automated guided vehicle with the fastest average speed in a predetermined time period among the plurality of sample delay factor state values as a reference automated guided vehicle; a first obtaining module, configured to obtain a delay step number of each of the automated guided vehicles other than the reference automated guided vehicle in a delay step number prediction period; and a fifth determining module, configured to determine a delay step number between each of the automated guided vehicles as the action data.

[0109] Optionally, the third determining subunit comprises: a sixth determining module, configured to determine the reward value by using a reward function, wherein the reward function is: R represents the reward value, represents an actual delay step number of the ith automated guided vehicle, A i represents a predicted delay step number of the ith automated guided vehicle, and n represents a total number of the automated guided vehicles.

[0110] Optionally, the training subunit comprises: a first adjusting module, configured to adjust the weight value by using the reward value through a weight value adjustment formula to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents an adjusted weight value of the jth sample delay factor state value of the ith automated guided vehicle, an initial weight value of a jth sample delay impact factor state value of an ith automated guided vehicle, R i an initial reward value of an ith automated guided vehicle, R k an initial reward value of an ith automated guided vehicle, R

[0111] Optionally, the training subunit comprises: a first adjustment module configured to adjust the weight value according to the reward value, so as to adjust the weight parameter of the delay step number prediction model; a second acquisition module configured to input the state data into the delay step number prediction model to obtain output data of the delay step number prediction model; and a stopping module configured to stop the training operation of the delay step number prediction model when a difference between the output data corresponding to the predicted action data and the action data is less than a preset difference value.

[0112] Optionally, the determination unit comprises: a fourth determination subunit configured to determine that the control strategy is to control each automated guided vehicle to operate in the current operating mode when the predicted delay step number is less than a delay step number threshold; and a fifth determination subunit configured to determine that the control strategy is to adjust all or part of the operating modes of the automated guided vehicles when the predicted delay step number is not less than the delay step number threshold.

[0113] According to another aspect of the embodiments of the present application, there is also provided an automated guided vehicle using the automated guided vehicle control method described in any one of the above.

[0114] According to another aspect of the embodiments of the present application, there is also provided an automated guided vehicle control system using the automated guided vehicle control method described in any one of the above.

[0115] According to another aspect of the embodiments of the present application, there is also provided a processor configured to run a program, wherein the program performs the automated guided vehicle control method described in any one of the above when the program is run.

[0116] According to another aspect of the embodiments of the present application, there is also provided a computer program product comprising computer instructions configured to perform the automated guided vehicle control method described in any one of the above when the computer instructions are executed by a processor.

[0117] According to another aspect of the embodiments of the present application, there is also provided a computer readable storage medium comprising a stored program, wherein the program performs the automated guided vehicle control method described in any one of the above.

[0118] Optionally, in the embodiment, the computer readable storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the communication devices in the communication device group.

[0119] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining delay factor state values corresponding to delay factors of the automated guided vehicles respectively, wherein the delay factors are factors causing the actual running speeds of the automated guided vehicles to be inconsistent with the planned running speeds, the planned running speeds are speeds pre-planned for the automated guided vehicles by a scheduling system of the automated guided vehicles, and the delay factor state values are numerical values corresponding to the delay factors; and inputting the delay factor state values into a delay step prediction model to process the delay factor state values by using the delay step prediction model to obtain predicted delay steps of the automated guided vehicles, wherein the predicted delay steps are used to evaluate speed difference values between the actual running speeds and the planned running speeds of the automated guided vehicles, and the delay step prediction model is a model trained by using a plurality of sets of training data through reinforcement learning.

[0120] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: determining the delay factors; obtaining current state information of the automated guided vehicles; and determining the delay factor state values corresponding to the delay factors of the automated guided vehicles respectively according to the current state information.

[0121] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: before inputting the delay factor state values into the delay step prediction model to process the delay factor state values by using the delay step prediction model to obtain the predicted delay steps of the automated guided vehicles, obtaining a plurality of sets of training data including a plurality of sample delay factor state values and a plurality of sample predicted delay steps corresponding to the plurality of sample delay factor state values; and training the plurality of sets of training data to obtain the delay step prediction model.

[0122] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a weight value of each of the plurality of sample delay factor state values; generating state data of the delay step prediction model according to the weight value of each of the plurality of sample delay factor state values and the plurality of sample delay factor state values; generating action data of the delay step prediction model according to each of the plurality of sample delay factor state values; determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; and training the delay step prediction model according to the state data, the action data, and the reward value.

[0123] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for determining the delay time influence degree of each automatic guided vehicle corresponding to the sample delay time influence factor state value; and determining the weight value according to the delay time influence degree.

[0124] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for generating a sample state matrix according to the plurality of sample delay time influence factor state values; generating a weight matrix according to the weight value; and determining the state data according to the Hadamard product of the state matrix and the weight matrix.

[0125] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for determining the automatic guided vehicle with the fastest average speed in the predetermined time period among the plurality of sample delay time influence factor state values as a reference automatic guided vehicle; obtaining the delay step number of each automatic guided vehicle except the reference automatic guided vehicle in the delay step number prediction period; and determining the delay step number between each automatic guided vehicle as the action data.

[0126] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for determining the reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay step number of the i th automatic guided vehicle, A i represents the predicted delay step number of the i th automatic guided vehicle, and n represents the total number of automatic guided vehicles.

[0127] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for adjusting the weight value by a weight value adjustment formula to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents the adjusted weight value of the j th sample delay time influence factor state value of the i th automatic guided vehicle, represents the initial weight value of the j th sample delay time influence factor state value of the i th automatic guided vehicle, R i represents the reward value of the i th automatic guided vehicle, R k represents the reward value of the k th automatic guided vehicle; and training the delay step number prediction model according to the state data, the action data, and the adjusted reward value.

[0128] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: adjusting the weight value according to the reward value to adjust the weight parameter of the delay step number prediction model; inputting the state data into the delay step number prediction model to obtain output data of the delay step number prediction model; and stopping the training operation of the delay step number prediction model when a difference between the predicted action data corresponding to the output data and the action data is less than a preset difference value.

[0129] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: determining the control strategy as controlling each automatic guided vehicle to operate in the current operation mode when the predicted delay step number is less than the delay step number threshold; and determining the control strategy as adjusting all or part of the operation mode of each automatic guided vehicle when the predicted delay step number is not less than the delay step number threshold.

[0130] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0131] In the above-mentioned embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other manners. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0133] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0134] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0135] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0136] The above description is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for controlling an automated guided vehicle, characterized in that, include: Obtain the delay influencing factors corresponding to the delay influencing factors of each automated guided vehicle (AGV). The delay influencing factors are the factors that cause the actual running speed of each AGV to be inconsistent with the planned running speed. The planned running speed is the speed pre-planned by the scheduling system of each AGV for each AGV. The delay influencing factor status value is the value corresponding to each delay influencing factor. Acquire multiple sets of training data, including the state values ​​of multiple sample delay influencing factors and the predicted delay steps of multiple samples corresponding to the state values ​​of multiple sample delay influencing factors; train the multiple sets of training data to obtain a delay step prediction model; The state values ​​of the delay influencing factors are input into the delay step prediction model to process the state values ​​of the delay influencing factors and obtain the predicted delay steps for each of the automated guided vehicles. The predicted delay steps are used to evaluate the speed difference between the actual running speed and the planned running speed of each of the automated guided vehicles. The delay step prediction model is a model trained by reinforcement learning using multiple sets of training data. The control strategy for each of the automated guided vehicles is determined based on the predicted delay steps. The process of training the multiple sets of training data to obtain the delay step prediction model includes: obtaining the weight value of each of the multiple sample delay influencing factor state values; generating state data of the delay step prediction model based on the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values; determining the Automated Guided Vehicle (AUV) with the fastest average speed within a predetermined time period corresponding to the multiple sample delay influencing factor state values ​​as the reference AUV; obtaining the delay steps of other vehicles in each AUV except the reference AUV within the delay step prediction period; determining the delay steps between each AUV as action data; determining a reward value through a reward function, wherein the reward function is: , R represents the reward value, represents the actual delay steps of the i-th AUV, represents the predicted delay steps of the i-th AUV, and represents the total number of AUVs, the reward value is used to optimize the weight values; and training the delay step prediction model based on the state data, the action data, and the reward value.

2. The automated guided vehicle control method according to claim 1, characterized in that, Obtain the state values ​​of the delay influencing factors for each automated guided vehicle, including: Identify each of the aforementioned delay-influencing factors; Obtain the current status information of each of the automated guided vehicles; Based on the current status information, determine the status value of the delay influencing factor corresponding to the delay influencing factor of each of the automated guided vehicles.

3. The automated guided vehicle control method according to claim 1, characterized in that, Obtain the weight value of each of the multiple sample delay influencing factor state values, including: Determine the degree of influence of the sample delay influencing factors corresponding to the state values ​​of the multiple sample delay influencing factors on the delay of each of the automated guided vehicles; The weight value is determined based on the degree of impact of the delay.

4. The automated guided vehicle control method according to claim 1, characterized in that, The state data of the delay step prediction model is generated based on the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values, including: A sample state matrix is ​​generated based on the state values ​​of multiple sample delay influencing factors. Generate a weight matrix based on the weight values; The state data is determined based on the Hadamard product of the state matrix and the weight matrix.

5. The automated guided vehicle control method according to claim 1, characterized in that, A delayed step prediction model is trained based on the state data, the action data, and the reward value, including: The weight value is adjusted using the reward value through a weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: , where represents the adjusted weight value of the j-th sample delay influencing factor state value of the i-th automated guided vehicle, represents the initial weight value of the j-th sample delay influencing factor state value of the i-th automated guided vehicle, represents the reward value of the i-th automated guided vehicle, and represents the reward value of the k-th automated guided vehicle; A delayed step prediction model is trained based on the state data, the action data, and the adjusted reward value.

6. The automated guided vehicle control method according to claim 1, characterized in that, A delayed step prediction model is trained based on the state data, the action data, and the reward value, including: The weight value is adjusted based on the reward value to adjust the weight parameters of the delayed step prediction model; The state data is input into the delay step prediction model to obtain the output data of the delay step prediction model; When the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference, the training operation of the delayed step prediction model is stopped.

7. The automated guided vehicle control method according to any one of claims 1 to 6, characterized in that, The control strategy for each of the automated guided vehicles is determined based on the predicted delay steps, including: When the predicted delay step number is less than the delay step threshold, the control strategy is determined to be: control each of the automated guided vehicles to operate according to the current operating mode; When the predicted delay step number is not less than the delay step number threshold, the control strategy is determined to be: to adjust the operating mode of all or part of the automated guided vehicles.

8. An automated guided vehicle (AGV) control device for implementing the AGV control method of claim 1, characterized in that, include: The first acquisition unit is used to acquire the delay influencing factor status value corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is the factor that causes the actual running speed of each automated guided vehicle to be inconsistent with the planned running speed, the planned running speed is the speed pre-planned by the scheduling system of each automated guided vehicle for each automated guided vehicle, and the delay influencing factor status value is the value corresponding to each delay influencing factor; The second acquisition unit is used to acquire multiple sets of training data, including the state values ​​of multiple sample delay influencing factors and the predicted delay steps of multiple samples corresponding to the state values ​​of multiple sample delay influencing factors; the training unit is used to train the multiple sets of training data to obtain a delay step prediction model. The processing unit is used to input the state values ​​of the delay influencing factors into the delay step prediction model, so as to process the state values ​​of the delay influencing factors using the delay step prediction model to obtain the predicted delay steps of each of the automated guided vehicles. The predicted delay steps are used to evaluate the speed difference between the actual running speed and the planned running speed of each of the automated guided vehicles. The delay step prediction model is a model trained by reinforcement learning using multiple sets of training data. A determining unit is used to determine the control strategy for each of the automated guided vehicles based on the predicted delay steps; The training unit includes: a second acquisition subunit for acquiring the weight value of each of the multiple sample delay influencing factor state values; a first generation subunit for generating state data of the delay step prediction model based on the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values; a second generation subunit for generating action data of the delay step prediction model based on each of the multiple sample delay influencing factor state values; a third determination subunit for determining a reward value based on the action data, wherein the reward value is used to optimize the weight value; and a training subunit for training the delay step prediction model based on the state data, the action data, and the reward value.

9. An automated guided vehicle, characterized in that, The automated guided vehicle uses the automated guided vehicle control method described in any one of claims 1 to 7.

10. An automated guided vehicle (AGV) control system, characterized in that, The automated guided vehicle control system uses the automated guided vehicle control method described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program executes the automated guided vehicle control method according to any one of claims 1 to 7.

12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the automated guided vehicle control method according to any one of claims 1 to 7 is performed.

Images