Automatic guided vehicle control method and device, automatic guided vehicle and control system thereof

Abstract

The invention discloses an automatic guided vehicle control method and device, an automatic guided vehicle and a control system of the automatic guided vehicle. The method comprises the steps that delay influence factor state values corresponding to delay influence factors of all the automatic guided vehicles are acquired, and the delay influence factors are factors causing the actual running speed and the planned running speed of all the automatic guided vehicles to be inconsistent; the planned running speed is the speed planned for each automatic guided vehicle in advance by the scheduling system of each automatic guided vehicle; inputting the state value of the delay influence factor into a delay step number prediction model, and processing the state value of the delay influence factor by using the delay step number prediction model to obtain a predicted delay step number of each automatic guided vehicle; and determining a control strategy of each automatic guided vehicle according to the predicted delay step number. According to the invention, the technical problems of poor reliability and potential safety hazard of the AGV scheduling system caused by disorder of the scheduling plan of the AGV system due to delay of part of AGVs in the related technology are solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to an automatic guided vehicle control method and device, an automatic guided vehicle and a control system thereof. Background Art

[0002] The automatic guided vehicle (AGV) automatic warehousing system usually includes modules such as task allocation and path planning. Among them, the path planning module is used to coordinate the planning of the execution task paths of all AGVs to avoid conflicts and complete the tasks efficiently. In the AGV scheduling system, the path planning algorithm usually assumes that all AGVs run at the same speed to plan conflict-free paths, and does not consider the speed differences caused by complex factors of each AGV in reality. In actual application scenarios, due to factors such as load and control differences between different AGVs, AGVs are delayed in path operation, making the actual running speed of AGVs not exactly the same as the speed planned by the system, resulting in asynchronous driving between each other. The delay of some AGVs will cause confusion in the scheduling plan, resulting in poor reliability of the scheduling system and inaccurate scheduling, and AGV conflicts will occur, causing economic losses and safety hazards.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide an automatic guided vehicle control method and device, an automatic guided vehicle and its control system, so as to at least solve the technical problem in the related technology that the delay of some AGVs will cause the scheduling plan of the AGV system to be disordered, making the reliability of the AGV scheduling system poor and posing a safety hazard.

[0005] According to one aspect of an embodiment of the present invention, there is provided an automated guided vehicle control method, comprising: obtaining delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is a factor causing the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; inputting the delay influencing factor state value into a delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain a predicted delay step number of each automated guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual operating speed of each automated guided vehicle and the planned operating speed, and the delay step prediction model is a model obtained by reinforcement learning training using multiple sets of training data; and determining a control strategy for each automated guided vehicle according to the predicted delay step number.

[0006] Optionally, obtaining the delay influencing factor state values ​​respectively corresponding to the delay influencing factors of each automatic guided vehicle includes: determining each of the delay influencing factors; obtaining current state information of each of the automatic guided vehicles; and determining the delay influencing factor state values ​​respectively corresponding to the delay influencing factors of each of the automatic guided vehicles based on the current state information.

[0007] Optionally, before inputting the delay influencing factor state value into the delay step prediction model to process the delay influencing factor state value using the delay step prediction model to obtain the predicted delay step number of each automatic guided vehicle, the automatic guided vehicle control method also includes: obtaining the multiple sets of training data including multiple sample delay influencing factor state values ​​and multiple sample predicted delay step numbers corresponding to the multiple sample delay influencing factor state values; training the multiple sets of training data to obtain the delay step prediction model.

[0008] Optionally, training the multiple groups of training data to obtain the delay step prediction model includes: obtaining a weight value of each of the multiple sample delay influencing factor state values; generating state data of the delay step prediction model according to the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values; generating action data of the delay step prediction model according to each of the multiple sample delay influencing factor state values; determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; and obtaining the delay step prediction model by training according to the state data, the action data and the reward value.

[0009] Optionally, obtaining a weight value of each of the multiple sample delay influencing factor state values ​​includes: determining the degree of delay influence of the sample delay influencing factors corresponding to the multiple sample delay influencing factor state values ​​on each of the automatic guided vehicles; and determining the weight value according to the delay influence degree.

[0010] Optionally, the state data of the delay step prediction model is generated according to the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values, including: generating a sample state matrix according to the multiple sample delay influencing factor state values; generating a weight matrix according to the weight values; and determining the state data according to the Hadamard product of the state matrix and the weight matrix.

[0011] Optionally, the action data of the delay step prediction model is generated according to each of the multiple sample delay influencing factor state values, including: determining the automated guided vehicle with the fastest average speed in the predetermined time period corresponding to the multiple sample delay influencing factor state values ​​as the reference automated guided vehicle; obtaining the delay steps of other vehicles among the automated guided vehicles except the reference automated guided vehicle within the delay step prediction period; and determining the delay steps between each of the automated guided vehicles as the action data.

[0012] Optionally, determining the reward value according to the action data includes: determining the reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th automatic guided vehicle, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of the automated guided vehicles.

[0013] Optionally, training a delay step prediction model according to the state data, the action data and the reward value includes: using the reward value to adjust the weight value through a weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the adjusted state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automatic guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k represents the reward value of the k-th automatic guided vehicle; and a delay step prediction model is obtained by training according to the state data, the action data and the adjusted reward value.

[0014] Optionally, a delay step prediction model is obtained by training according to the state data, the action data and the reward value, including: adjusting the weight value according to the reward value to adjust the weight parameters of the delay step prediction model; inputting the state data into the delay step prediction model to obtain output data of the delay step prediction model; and stopping the training operation of the delay step prediction model when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference.

[0015] Optionally, the control strategy of each of the automatic guided vehicles is determined according to the predicted delay step number, including: when the predicted delay step number is less than the delay step number threshold, determining the control strategy as: controlling each of the automatic guided vehicles to operate according to the current operating mode; when the predicted delay step number is not less than the delay step number threshold, determining the control strategy as: adjusting all or part of the operating mode of each of the automatic guided vehicles.

[0016] According to another aspect of an embodiment of the present invention, an automatic guided vehicle control device is further provided, comprising: a first acquisition unit, configured to acquire delay influencing factor state values ​​corresponding to the delay influencing factors of each automatic guided vehicle, wherein the delay influencing factor is a factor causing the actual operating speed of each automatic guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is a speed pre-planned for each automatic guided vehicle by a scheduling system of each automatic guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; a processing unit, configured to input the delay influencing factor state value into a delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain a predicted delay step number of each automatic guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual operating speed of each automatic guided vehicle and the planned operating speed, and the delay step prediction model is a model obtained by reinforcement learning training using multiple sets of training data; a determination unit, configured to determine the control strategy of each automatic guided vehicle according to the predicted delay step number.

[0017] Optionally, the first acquisition unit includes: a first determination subunit, used to determine each of the delay influencing factors; a first acquisition subunit, used to obtain current status information of each of the automatic guided vehicles; and a second determination subunit, used to determine the delay influencing factor status values ​​corresponding to the delay influencing factors of each of the automatic guided vehicles based on the current status information.

[0018] Optionally, the automatic guided vehicle control device also includes: a second acquisition unit, used to obtain the multiple groups of training data including multiple sample delay influencing factor state values ​​and multiple sample predicted delay steps corresponding to the multiple sample delay influencing factor state values ​​before inputting the delay influencing factor state value into the delay step prediction model to process the delay influencing factor state value using the delay step prediction model to obtain the predicted delay steps of each automatic guided vehicle; a training unit, used to train the multiple groups of training data to obtain the delay step prediction model.

[0019] Optionally, the training unit includes: a second acquisition subunit, used to obtain the weight value of each of the multiple sample delay influencing factor state values; a first generation subunit, used to generate state data of the delay step prediction model according to the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values; a second generation subunit, used to generate action data of the delay step prediction model according to each of the multiple sample delay influencing factor state values; a third determination subunit, used to determine a reward value according to the action data, wherein the reward value is used to optimize the weight value; a training subunit, used to train the delay step prediction model according to the state data, the action data and the reward value.

[0020] Optionally, the second acquisition subunit includes: a first determination module, used to determine the degree of delay influence of the sample delay influencing factors corresponding to the multiple sample delay influencing factor state values ​​on each of the automatic guided vehicles; and a second determination module, used to determine the weight value according to the delay influence degree.

[0021] Optionally, the first generating subunit includes: a first generating module, used to generate a sample state matrix according to the state values ​​of multiple sample delay influencing factors; a second generating module, used to generate a weight matrix according to the weight values; and a third determining module, used to determine the state data according to the Hadamard product of the state matrix and the weight matrix.

[0022] Optionally, the second generating sub-unit includes: a fourth determination module, used to determine the automated guided vehicle with the fastest average speed in a predetermined time period corresponding to a plurality of the sample delay influencing factor state values ​​as a reference automated guided vehicle; a first acquisition module, used to obtain the delay steps of other vehicles among the automated guided vehicles except the reference automated guided vehicle within a delay step prediction period; and a fifth determination module, used to determine the delay steps between each of the automated guided vehicles as the action data.

[0023] Optionally, the third determination subunit includes: a sixth determination module, configured to determine the reward value by using a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th automatic guided vehicle, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of the automated guided vehicles.

[0024] Optionally, the training subunit includes: a first adjustment module, configured to adjust the weight value by using the reward value through a weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the adjusted state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automatic guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k Represents the reward value of the k-th automatic guided vehicle; a training module, used to train a delay step prediction model according to the state data, the action data and the adjusted reward value.

[0025] Optionally, the training subunit includes: a first adjustment module, used to adjust the weight value according to the reward value to adjust the weight parameters of the delay step prediction model; a second acquisition module, used to input the state data into the delay step prediction model to obtain output data of the delay step prediction model; and a stop module, used to stop the training operation of the delay step prediction model when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference.

[0026] Optionally, the determination unit includes: a fourth determination subunit, used to determine that the control strategy is: controlling each of the automatic guided vehicles to operate according to the current operating mode when the predicted delay step number is less than the delay step number threshold; a fifth determination subunit, used to determine that the control strategy is: adjusting all or part of the operating mode of each of the automatic guided vehicles when the predicted delay step number is not less than the delay step number threshold.

[0027] According to another aspect of the embodiments of the present invention, an automatic guided vehicle is provided. The automatic guided vehicle uses any one of the automatic guided vehicle control methods described above.

[0028] According to another aspect of an embodiment of the present invention, an automatic guided vehicle control system is further provided, wherein the automatic guided vehicle control system uses any one of the automatic guided vehicle control methods described above.

[0029] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the above-mentioned automatic guided vehicle control methods.

[0030] According to another aspect of an embodiment of the present invention, a processor is further provided, wherein the processor is used to run a program, wherein the program executes any one of the above-mentioned automatic guided vehicle control methods when running.

[0031] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, any one of the above-mentioned automatic guided vehicle control methods is executed.

[0032] In an embodiment of the present invention, delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle are obtained, wherein the delay influencing factor is a factor causing the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned for each automated guided vehicle by the scheduling system of each automated guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; the delay influencing factor state value is input into a delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain a predicted delay step number of each automated guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual operating speed of each automated guided vehicle and the planned operating speed, and the delay step prediction model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample delay influencing factor state value and a sample predicted delay step number corresponding to the sample delay influencing factor state value; and a control strategy for each automated guided vehicle is determined according to the predicted delay step number. Through the above-mentioned technical scheme provided by the present invention, it is achieved that a delay step prediction model is pre-trained, so that after the state value of the delay influencing factor of each AGV is obtained, the predicted delay step number of each AGV is determined by using the delay step prediction model, and then the predicted delay step number is used as the factor to control the operation of each AGV, thereby achieving the technical effect of accurately predicting the operation steps of the AGV, so that the relative delay between AGVs can be obtained, so that the actual and theoretical planned paths of the AGVs in the actual application scenario are consistent, and accurate prediction of the AGV scheduling delay is achieved, thereby solving the technical problem in the related technology that the delay of some AGVs will cause the scheduling plan of the AGV system to be disordered, resulting in poor reliability of the AGV scheduling system and potential safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0034] Figure 1 It is a hardware structure block diagram of a mobile terminal of an automatic guided vehicle control method according to an embodiment of the present invention;

[0035] Figure 2 is a flow chart of an automatic guided vehicle control method according to an embodiment of the present invention;

[0036] Figure 3 is a flow chart of an optional automatic guided vehicle control method according to an embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of an automatic guided vehicle control device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] As introduced in the background technology, the delay of some AGVs in the related technology will cause the scheduling plan of the AGV system to be disordered, making the reliability of the AGV scheduling system poor and posing a safety hazard. In view of the above defects, an automatic guided vehicle control method and device, a computer-readable storage medium, a processor and a computer program product are provided in the embodiments of the present invention.

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0042] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of an automatic guided vehicle control method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0043] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the automatic guided vehicle control method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0044] According to an embodiment of the present invention, a method embodiment of an automatic guided vehicle control method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Figure 2 is a flow chart of an automatic guided vehicle control method according to an embodiment of the present invention. Figure 2 As shown, the automatic guided vehicle control method includes the following steps:

[0046] Step S202, obtaining the delay influencing factor status values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factors are factors that cause the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned by the scheduling system of each automated guided vehicle for each automated guided vehicle, and the delay influencing factor status values ​​are the numerical values ​​corresponding to each delay influencing factor.

[0047] In the embodiment of the present invention, when the AGV scheduling system performs path planning for each AGV, it will regard the AGVs in the AVG scheduling system as being exactly the same, and when scheduling and planning the AVG, in the theoretical running path, the running speed set by the AGV will be regarded as completely uniform. In the actual running process, the AGV will inevitably appear in the states of starting acceleration, stable running, deceleration and stopping, and only when it is running smoothly, its actual running speed is the same as the theoretically set running speed. Therefore, for the path planning algorithm of multi-AGV collaborative planning, in the actual application scenario, there is a conflict-free path planned in theory, but in the actual operation, conflicts are caused because all AGVs cannot be completely synchronized.

[0048] The inability of AGVs to be fully synchronized is caused by a variety of factors, such as load. The heavier the load, the slower the speed; and the age and brand of the vehicle. The acceleration and deceleration of each AGV are different, resulting in speed differences. In addition, different distances from the target point will also trigger different speed control mechanisms of the controller. For example, the closer the distance to the target point, the greater the deceleration may be, ensuring that the AGV stops in time at the target point.

[0049] Here, five factors including load, acceleration, deceleration, current speed and distance from the target point can be selected as delay prediction parameters, that is, delay influencing factors.

[0050] It should be noted that the above delay influencing factors are not limited to these, and different influencing factors can be considered in applications according to actual conditions in different scenarios.

[0051] The above delay influencing factor state value may be the value corresponding to each delay influencing factor during the actual operation of each AGV, for example, the AGV load value f i , acceleration f i , deceleration r i , current speed v i , the distance from the target point is d i .

[0052] Step S204, inputting the delay influencing factor state value into the delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain the predicted delay step number of each automatic guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, and the delay step prediction model is a model obtained by machine learning training using multiple sets of training data, each set of training data in the multiple sets of training data includes: a sample delay influencing factor state value and a sample predicted delay step number corresponding to the sample influencing factor state value, and an actual step number corresponding to the sample delay influencing factor state value, a reward function is constructed based on the predicted delay step number and the actual step number, and the model is optimized according to the reward function.

[0053] Here, the delay influencing factor state values ​​obtained in the above steps can be input into a pre-trained delay step prediction model, and the delay step prediction model is used to process the above delay influencing factor state values, so as to obtain the predicted delay steps of each AGV, and then determine the delay steps of each AGV within the predicted steps.

[0054] Step S206, determining the control strategy of each automatic guided vehicle according to the predicted delay step number.

[0055] In this embodiment, the operation strategy of each AGV, that is, the above-mentioned control strategy, can be determined according to the predicted delay steps obtained by the above prediction. Among them, the control strategy here can be determined according to the specific delay steps obtained by the above prediction. For example, the control strategy can be to predict the delay steps to determine whether the automatic guided vehicle needs to accelerate (when there are no other vehicles in front of the automatic guided vehicle and there will be no conflict between the vehicles, if the predicted delay steps are large, the automatic guided vehicle can be controlled to accelerate; on the contrary, if there may be a conflict with the vehicle in front according to the current speed, the driving speed of the automatic guided vehicle can be calculated according to the delay steps to determine that the automatic guided vehicle will neither conflict with the vehicle in front nor affect the normal driving of the rear vehicle at this driving speed).

[0056] In the embodiment of the present invention, the predicted number of steps is to ensure that the AGVs operate in a coordinated and orderly manner according to the system settings, neither too fast nor too slow, that is, accurately, so as to prevent scheduling confusion and indirectly prevent collisions.

[0057] In addition, after the control strategy of each automated guided vehicle is determined through the above steps, each automated guided vehicle can be controlled to operate according to the control strategy to ensure that the automated guided vehicles can operate in an orderly manner.

[0058] From the above, it can be seen that in an embodiment of the present invention, the delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle are obtained, wherein the delay influencing factor is a factor that causes the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned for each automated guided vehicle by the scheduling system of each automated guided vehicle, and the delay influencing factor state value is the numerical value corresponding to each delay influencing factor; the delay influencing factor state value is input into the delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain the predicted delay step number of each automated guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual operating speed of each automated guided vehicle and the planned operating speed, and the delay step prediction model is a method of using multiple sets of training data to pass The model obtained through machine learning training, each of the multiple groups of training data includes: a sample delay influencing factor state value and a sample predicted delay step number corresponding to the sample delay influencing factor state value; the control strategy of each automatic guided vehicle is determined according to the predicted delay step number, and the delay step number prediction model is pre-trained. After obtaining the delay influencing factor state value of each AGV, the predicted delay step number of each AGV is determined by using the delay step number prediction model, and then the predicted delay step number is used as a factor for controlling the operation of each AGV, thereby achieving the purpose of accurately predicting the operation steps of the AGV, so as to obtain the relative delay between AGVs, so as to achieve the technical effect that the actual AGV in the actual application scenario is consistent with the theoretical planned path, and the accurate prediction of the AGV scheduling delay is achieved.

[0059] Therefore, the above technical solution provided by the embodiment of the present invention solves the technical problem in the related art that the delay of some AGVs may cause the scheduling plan of the AGV system to be disordered, resulting in poor reliability of the AGV scheduling system and potential safety hazards.

[0060] According to the above embodiment of the present invention, obtaining the delay influencing factor status values ​​corresponding to the delay influencing factors of each automatic guided vehicle may include: determining each delay influencing factor; obtaining the current status information of each automatic guided vehicle; and determining the delay influencing factor status values ​​corresponding to the delay influencing factors of each automatic guided vehicle based on the current status information.

[0061] In this embodiment, the current state information of each AGV, such as load, acceleration, deceleration, current speed, and distance from the target point, can be obtained, so that the delay influencing factor state value of each delay influencing factor can be determined according to the values ​​of these factors.

[0062] According to the above-mentioned embodiment of the present invention, before the delay influencing factor state value is input into the delay step number prediction model to process the delay influencing factor state value using the delay step number prediction model to obtain the predicted delay step number of each automatic guided vehicle, the automatic guided vehicle control method also includes: obtaining multiple groups of training data including sample delay influencing factor state value and sample predicted delay step number corresponding to the sample influencing factor state value, and actual step number corresponding to the sample delay influencing factor state value; training the multiple groups of training data to obtain the delay step number prediction model.

[0063] In this embodiment, the historical operation data of the above-mentioned AGVs in historical time can be obtained, and these historical operation data can be analyzed to obtain the above-mentioned groups of training data, that is, the state value of the sample delay influencing factor and the sample predicted delay step number corresponding to the state value of the sample delay influencing factor.

[0064] According to the above embodiment of the present invention, training multiple groups of training data to obtain a delay step prediction model can include: obtaining a weight value of each of a plurality of sample delay influencing factor state values; generating state data of the delay step prediction model according to the weight value of each of a plurality of sample delay influencing factor state values ​​and a plurality of sample delay influencing factor state values; generating action data of the delay step prediction model according to each of a plurality of sample delay influencing factor state values; determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; and obtaining the delay step prediction model by training according to the state data, the action data, and the reward value.

[0065] In this embodiment, a weight value of each of the multiple sample delay influencing factor state values ​​may be obtained, and the weight value is used to characterize the degree of influence of each sample delay influencing factor state value on the delay generated by each AGV.

[0066] In addition, in the embodiment of the present invention, the core elements of reinforcement learning are: state, action and reward function, which are the key to modeling. The above core elements will be introduced in detail below and will not be repeated here.

[0067] In the above embodiment, obtaining the weight value of each of the multiple sample delay influencing factor state values ​​includes: determining the delay influence degree of the sample delay influencing factors corresponding to the multiple sample delay influencing factor state values ​​on each automatic guided vehicle; and determining the weight value according to the delay influence degree.

[0068] In actual AGV scheduling, each influencing factor has different effects on the AGV operation delay, so the factor weight value here is used to determine the influence of each AGV's state factor on the AGV operation delay.

[0069] In the above embodiment, state data of the delay step prediction model is generated according to the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values, including: generating a sample state matrix according to the multiple sample delay influencing factor state values; generating a weight matrix according to the weight values; and determining the state data according to the Hadamard product of the state matrix and the weight matrix.

[0070] In this embodiment, a sample state matrix can be generated according to the state values ​​of multiple sample delay influencing factors, such as state: AGV load value f i , acceleration a i , deceleration r i 、Current speed v i , distance d from the target point i Assuming three AGVs are considered, the state matrix can be expressed as:

[0071] In actual AGV scheduling, each influencing factor has different effects on AGV operation delay, so a weight matrix is ​​introduced here to set the degree of influence of each AGV's state factor on AGV operation delay. Each element of the weight matrix corresponds to the weight of each state parameter.

[0072] Next, based on the state matrix and the weight matrix, the model input can be set as the Hadamard product of the state matrix S and the weight matrix W, that is, the corresponding elements of the two matrices are multiplied. Therefore, the model input is expressed as:

[0073] Each element of the W matrix can be initialized to 1 (by default, all influencing factors of all AGVs have the same weight), or set to a specific value based on the characteristics of each AGV, and then dynamically adjusted according to the reward function during the model optimization process.

[0074] According to the above embodiment of the present invention, the action data of the delay step prediction model is generated according to each of the multiple sample delay influencing factor state values, including: determining the automated guided vehicle with the fastest average speed in the predetermined time period corresponding to the multiple sample delay influencing factor state values ​​as the reference automated guided vehicle; obtaining the delay steps of the other vehicles in the automated guided vehicles except the reference automated guided vehicle within the delay step prediction period; and determining the delay steps between the automated guided vehicles as the action data.

[0075] In this embodiment, the action data can be set according to the number of steps of the AGV. Since the synchronization between AGVs is relative, that is, if we set the AGV with the fastest average speed over a period of time as the standard, the running status of other AGVs can be expressed as a delay relative to the fastest AGV. Therefore, we set the action as the relative delay steps between AGVs.

[0076] Assuming that five steps are used as a prediction cycle, the maximum delay is four steps. 0 means that the AGV has the fastest speed and the delay step is 0, -1 means that the relative delay is 1 step relative to the fastest AGV, -2 means that the relative delay is 2 steps relative to the fastest AGV, and so on. There are several action options. It means that one of the three AGVs must have the fastest speed, that is, the action is 0; It means that for the remaining two AGVs, each AGV's action is selected from the five choices (0, -1, -2, -3, -4). For example, taking three AGVs as an example, the action (-10-3) means that AGV2 has the fastest average speed in the next 5 steps. Based on it, AGV1 will be relatively delayed by 1 step, and AGV3 will be relatively delayed by 3 steps.

[0077] According to the above embodiment of the present invention, determining the reward value according to the action data includes: determining the reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th AGV, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of automated guided vehicles.

[0078] Here, the reward function can be set to the actual number of delay steps of the AGV over a period of time The number of delay steps with the prediction A i The inverse of the mean square error of : Where n represents the number of AGVs. The R value is used as a reward function to update the model training parameters.

[0079] According to the above embodiment of the present invention, a delay step prediction model is obtained according to the state data, action data and reward value training, including: using the reward value to adjust the weight value through the weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the state value of the delay influencing factor of the jth sample of the i-th automated guided vehicle after adjustment, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k represents the reward value of the k-th automated guided vehicle; a delay step prediction model is trained based on the state data, action data and adjusted reward value.

[0080] Since the conditions of AGVs in practice are not exactly the same, that is, the actual delay steps of one or several AGVs may differ greatly from the predicted delay steps. If only the expression form of reward R is used, the prediction effect of other AGVs whose predicted steps are consistent with the actual steps will not be good under the training of reward function R. Therefore, in the embodiment of the present invention, the weight of each individual AGV is designed, specifically, R corresponding to each AGV i The value is used alone as the reference value of the weight matrix input above, which is expressed as the above weight value adjustment formula.

[0081] in, is the initial value of the weight matrix element (all elements are 1). Based on this expression, the reward and punishment of each AGV can be fed back to the weight parameters of the trained network, and the network can be trained with focus. To a certain extent, it can avoid the poor prediction performance of individual AGVs affecting the convergence speed of the entire model to achieve a better strategy, thereby making the prediction effect of AGV better.

[0082] According to the above embodiment of the present invention, a delay step prediction model is obtained according to state data, action data and reward value training, including: adjusting the weight value according to the reward value to adjust the weight parameter of the delay step prediction model; inputting the state data into the delay step prediction model to obtain output data of the delay step prediction model; when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference, stopping the training operation of the delay step prediction model.

[0083] In this embodiment, the weight value can be adjusted according to the above-mentioned reward value, that is, the reward and punishment situation of each AGV is fed back to the weight parameters of the trained network, and the network can be trained in a focused manner. To a certain extent, it can avoid the poor prediction performance of individual AGVs affecting the convergence speed of the entire model to achieve a better strategy, thereby making the prediction effect of AGV better.

[0084] In the above steps, according to the general reinforcement learning training steps, after initialization, the AGV state parameter S at time t is t Input the reinforcement learning algorithm to obtain the action parameter A at time t t , based on the action parameter path planning algorithm, a conflict-free path with a set number of steps is planned and sent to the AGV.

[0085] When an AGV runs to the set number of steps (i.e., the delay prediction period, how many steps are used to predict the delay steps), the delay steps of each AGV are counted, and the reward function R at time (t+1) is calculated accordingly. t+1 , and input the state parameter S at time (t+1) t+1, update the model parameters according to the specific reinforcement learning algorithm strategy. The above method can be repeated until the delay prediction mode converges and the model training is completed.

[0086] According to the above embodiment of the present invention, the control strategy of each automatic guided vehicle is determined according to the predicted delay step number, including: when the predicted delay step number is less than the delay step number threshold, the control strategy is determined as: controlling each automatic guided vehicle to operate according to the current operating mode; when the predicted delay step number is not less than the delay step number threshold, the control strategy is determined as: adjusting all or part of the operating modes of each automatic guided vehicle.

[0087] In this embodiment, the operation of each AGV can be controlled based on the relationship between the predicted delay step number and the delay step number threshold (i.e., the maximum allowable delay step number, within which the AGV will not collide) to prevent collision between the AGVs.

[0088] For example, when the delay step threshold is 5 steps and the predicted delay step is 3 steps, then each automated guided vehicle can be controlled without adjusting the current operating speed and route, that is, it can operate according to the current operating mode; on the contrary, if the predicted delay step is 8 steps, in this case, it may affect the operation of subsequent automated guided vehicles. At this time, the operating mode can be adjusted according to the current operating speed and route of each automated guided vehicle to ensure that the automated guided vehicles can operate in an orderly manner.

[0089] Figure 3 is a flow chart of an optional automatic guided vehicle control method according to an embodiment of the present invention, such as Figure 3 As shown, first, you can set the state, action and reward function, then input the state, get the action and regulate the AGV operation; then count the real-time delay steps, calculate the reward function and update the state; then update the model parameters, and when the model reaches convergence, save the model; in actual applications, call the model to predict the delay steps.

[0090] In addition, in actual applications, you only need to call the trained model for real-time prediction, or continue training the original trained model based on new data, and make it more suitable for the application scenario through parameter feedback.

[0091] Through the above technical solution provided by the embodiment of the present invention, based on reinforcement learning technology, by considering various delay influencing factors of AGV (such as load, acceleration, deceleration, current speed and distance from the target point), and dynamically adjusting the factor weight matrix according to the difference between the delay prediction and the actual delay of each AGV, the convergence of the model is accelerated. Based on the delay situation, the path planning algorithm can plan a conflict-free path more accurately, which improves the operating efficiency of AGV at the source.

[0092] Therefore, the technical solution provided by the embodiment of the present invention solves the AGV conflict problem caused by the inconsistency between the actual and theoretical AGVs, and improves the system reliability and production safety. It adopts a reinforcement learning algorithm (such as Double Deep Q Network (DDQN)) to predict the relative delay steps between AGVs within a period of time as the basis for AGV path planning, takes the AGV load value, acceleration, deceleration, current speed and distance from the target point as state parameters, takes the relative delay steps between AGVs as action parameters, and takes the inverse of the mean square error between the actual relative delay steps of AGVs and the predicted relative delay steps during this period as a reward function to guide the algorithm to train, and according to the difference between AGVs, the reward function of each AGV is used to feedback the adjustment factor weight matrix, accurately predict the running steps of AGVs, so as to obtain the relative delay between AGVs, so that the actual AGV in the actual application scenario is consistent with the theoretical planning path, solves the AGV conflict problem, improves the system reliability, and ensures the safety of actual production and transportation. At the same time, it also solves the AGV conflict problem caused by delay when AGVs are not synchronized in actual application scenarios, predicts the delay of multiple AGVs, enables the path planning algorithm to achieve more accurate scheduling of the actual operation of AGVs, avoids the conflict problem of AGVs caused by delays, improves the operating efficiency of the AGV scheduling system, and ensures the safety of actual production.

[0093] That is, a reinforcement learning algorithm is used to predict the relative delay steps between AGVs within a period of time as the basis for AGV path planning. The AGV load value, acceleration, deceleration, current speed and distance to the target point are used as state parameters, and the relative delay steps between AGVs are used as action parameters. The inverse of the mean square error between the actual relative delay steps of AGVs and the predicted relative delay steps during this period is used as a reward function to guide the algorithm training. According to the differences between AGVs, the reward function of each AGV is used to feedback the weight matrix of the adjustment factors to accurately predict the running steps of the AGV, so as to obtain the relative delay between AGVs, so that the actual and theoretical planned paths of AGVs in actual application scenarios are consistent, and accurate prediction of AGV scheduling delay is achieved.

[0094] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0096] According to an embodiment of the present invention, there is also provided an automatic guided vehicle control device for implementing the automatic guided vehicle control method. Figure 4 is a schematic diagram of an automatic guided vehicle control device according to an embodiment of the present invention. Figure 4 As shown, the device includes: a first acquisition unit 401, a processing unit 403 and a determination unit 407. The automatic guided vehicle control device is described in detail below.

[0097] The first acquisition unit 401 is used to obtain the delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is a factor that causes the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned by the scheduling system of each automated guided vehicle for each automated guided vehicle, and the delay influencing factor state value is the numerical value corresponding to each delay influencing factor.

[0098] The processing unit 403 is used to input the state value of the delay influencing factor into the delay step prediction model, so as to use the delay step prediction model to process the state value of the delay influencing factor to obtain the predicted delay step number of each automatic guided vehicle, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each automatic guided vehicle, and the delay step prediction model is a model obtained by reinforcement learning training using multiple sets of training data.

[0099] The determination unit 407 is used to determine the control strategy of each automatic guided vehicle according to the predicted delay step number.

[0100] It should be noted here that the above-mentioned first acquisition unit 401, processing unit 403 and determination unit 407 correspond to steps S202 to S206 in the above-mentioned embodiment, and the three units and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned embodiment.

[0101] It can be seen from the above that in the scheme recorded in the above-mentioned embodiment of the present invention, the first acquisition unit can be used to obtain the delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is the factor that causes the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned for each automated guided vehicle by the scheduling system of each automated guided vehicle, and the delay influencing factor state value is the numerical value corresponding to each delay influencing factor; the processing unit is used to input the delay influencing factor state value into the delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain the predicted delay step number of each automated guided vehicle, wherein the predicted delay step number is used to evaluate the actual operation of each automated guided vehicle. The speed difference between the speed and the planned running speed, the delay step prediction model is a model obtained by reinforcement learning training using multiple sets of training data; the determination unit is used to determine the control strategy of each automatic guided vehicle according to the predicted delay step, and the delay step prediction model is pre-trained to obtain the state value of the delay influencing factor of each AGV. After that, the predicted delay step is used to determine the predicted delay step of each AGV, and then the predicted delay step is used as the factor for controlling the operation of each AGV, so as to accurately predict the running steps of the AGV, so as to obtain the relative delay between the AGVs, so that the actual AGV in the actual application scenario is consistent with the theoretical planned path. The technical effect achieves accurate prediction of the AGV scheduling delay.

[0102] Therefore, the above technical solution provided by the embodiment of the present invention solves the technical problem in the related art that the delay of some AGVs may cause the scheduling plan of the AGV system to be disordered, resulting in poor reliability of the AGV scheduling system and potential safety hazards.

[0103] Optionally, the first acquisition unit includes: a first determination subunit, used to determine each delay influencing factor; a first acquisition subunit, used to obtain current status information of each automatic guided vehicle; and a second determination subunit, used to determine the delay influencing factor status values ​​corresponding to the delay influencing factors of each automatic guided vehicle based on the current status information.

[0104] Optionally, the automatic guided vehicle control device also includes: a second acquisition unit, which is used to obtain multiple groups of training data including multiple sample delay influencing factor state values ​​and multiple sample predicted delay steps corresponding to the multiple sample delay influencing factor state values ​​before inputting the delay influencing factor state value into the delay step number prediction model to process the delay influencing factor state value using the delay step number prediction model to obtain the predicted delay steps of each automatic guided vehicle; a training unit, which is used to train the multiple groups of training data to obtain the delay step number prediction model.

[0105] Optionally, the training unit includes: a second acquisition subunit, used to obtain the weight value of each of the multiple sample delay influencing factor state values; a first generation subunit, used to generate state data of the delay step prediction model according to the weight value of each of the multiple sample delay influencing factor state values ​​and the multiple sample delay influencing factor state values; a second generation subunit, used to generate action data of the delay step prediction model according to each of the multiple sample delay influencing factor state values; a third determination subunit, used to determine the reward value according to the action data, wherein the reward value is used to optimize the weight value; and a training subunit, used to train the delay step prediction model according to the state data, action data and reward value.

[0106] Optionally, the second acquisition subunit includes: a first determination module, used to determine the delay influence degree of the sample delay influencing factors corresponding to the state values ​​of multiple sample delay influencing factors on each automatic guided vehicle; and a second determination module, used to determine the weight value according to the delay influence degree.

[0107] Optionally, the first generating subunit includes: a first generating module, used to generate a sample state matrix according to the state values ​​of multiple sample delay influencing factors; a second generating module, used to generate a weight matrix according to the weight value; and a third determining module, used to determine the state data according to the Hadamard product of the state matrix and the weight matrix.

[0108] Optionally, the second generating subunit includes: a fourth determination module, used to determine the automated guided vehicle with the fastest average speed in a predetermined time period corresponding to multiple sample delay influencing factor state values ​​as a reference automated guided vehicle; a first acquisition module, used to obtain the delay steps of other vehicles among the automated guided vehicles except the reference automated guided vehicle within the delay step prediction period; and a fifth determination module, used to determine the delay steps between each automated guided vehicle as action data.

[0109] Optionally, the third determination subunit includes: a sixth determination module, configured to determine the reward value by using a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th AGV, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of automated guided vehicles.

[0110] Optionally, the training subunit includes: a first adjustment module, configured to adjust the weight value by using the reward value through a weight value adjustment formula to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the state value of the delay influencing factor of the jth sample of the i-th automated guided vehicle after adjustment, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k represents the reward value of the k-th automated guided vehicle; a training module is used to train a delay step prediction model based on the state data, action data and adjusted reward value.

[0111] Optionally, the training subunit includes: a first adjustment module, used to adjust the weight value according to the reward value to adjust the weight parameter of the delay step prediction model; a second acquisition module, used to input the state data into the delay step prediction model to obtain the output data of the delay step prediction model; and a stop module, used to stop the training operation of the delay step prediction model when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference.

[0112] Optionally, the determination unit includes: a fourth determination subunit, used to determine the control strategy as: controlling each automatic guided vehicle to operate according to the current operating mode when the predicted delay step number is less than the delay step number threshold; a fifth determination subunit, used to determine the control strategy as: adjusting all or part of the operating mode of each automatic guided vehicle when the predicted delay step number is not less than the delay step number threshold.

[0113] According to another aspect of an embodiment of the present invention, an automatic guided vehicle is provided. The automatic guided vehicle uses any one of the automatic guided vehicle control methods described above.

[0114] According to another aspect of an embodiment of the present invention, an automatic guided vehicle control system is provided. The automatic guided vehicle control system uses any one of the automatic guided vehicle control methods described above.

[0115] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, any one of the above-mentioned automatic guided vehicle control methods is executed.

[0116] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer instructions, and when the computer instructions are executed by a processor, any one of the above-mentioned automatic guided vehicle control methods is executed.

[0117] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes any one of the above-mentioned automatic guided vehicle control methods.

[0118] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0119] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining the delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is a factor that causes the actual operating speed of each automated guided vehicle to be inconsistent with the planned operating speed, the planned operating speed is the speed pre-planned for each automated guided vehicle by the scheduling system of each automated guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; inputting the delay influencing factor state value into a delay step prediction model, so as to process the delay influencing factor state value using the delay step prediction model to obtain a predicted delay step of each automated guided vehicle, wherein the predicted delay step is used to evaluate the speed difference between the actual operating speed of each automated guided vehicle and the planned operating speed, and the delay step prediction model is a model trained by reinforcement learning using multiple sets of training data.

[0120] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: determining each delay influencing factor; obtaining current status information of each automatic guided vehicle; and determining the delay influencing factor status values ​​corresponding to the delay influencing factors of each automatic guided vehicle based on the current status information.

[0121] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: before inputting the delay influencing factor state value into the delay step number prediction model to process the delay influencing factor state value using the delay step number prediction model to obtain the predicted delay step number of each automatic guided vehicle, obtaining multiple sets of training data including multiple sample delay influencing factor state values ​​and multiple sample predicted delay step numbers corresponding to the multiple sample delay influencing factor state values; training the multiple sets of training data to obtain the delay step number prediction model.

[0122] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining a weight value for each of a plurality of sample delay influencing factor state values; generating state data of a delay step prediction model based on the weight value for each of a plurality of sample delay influencing factor state values ​​and a plurality of sample delay influencing factor state values; generating action data of a delay step prediction model based on each of a plurality of sample delay influencing factor state values; determining a reward value based on the action data, wherein the reward value is used to optimize the weight value; and obtaining a delay step prediction model by training based on the state data, the action data, and the reward value.

[0123] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: determining the degree of delay influence of sample delay influencing factors corresponding to multiple sample delay influencing factor state values ​​on each automatic guided vehicle; and determining a weight value according to the degree of delay influence.

[0124] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: generating a sample state matrix based on state values ​​of multiple sample delay influencing factors; generating a weight matrix based on weight values; and determining state data based on the Hadamard product of the state matrix and the weight matrix.

[0125] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: determining the automated guided vehicle with the fastest average speed in a predetermined time period corresponding to multiple sample delay influencing factor state values ​​as a reference automated guided vehicle; obtaining the number of delay steps of other automated guided vehicles except the reference automated guided vehicle within a delay step prediction period; and determining the number of delay steps between each automated guided vehicle as action data.

[0126] Optionally, in this embodiment, the computer-readable storage medium is configured to store a program code for performing the following steps: determining a reward value by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th AGV, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of automated guided vehicles.

[0127] Optionally, in this embodiment, the computer-readable storage medium is configured to store a program code for executing the following steps: using the reward value to adjust the weight value through a weight value adjustment formula to obtain an adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the state value of the delay influencing factor of the jth sample of the i-th automated guided vehicle after adjustment, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k represents the reward value of the k-th automated guided vehicle; a delay step prediction model is trained based on the state data, action data and adjusted reward value.

[0128] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: adjusting the weight value according to the reward value to adjust the weight parameters of the delay step prediction model; inputting the state data into the delay step prediction model to obtain the output data of the delay step prediction model; when the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference, stopping the training operation of the delay step prediction model.

[0129] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: when the predicted delay step number is less than the delay step number threshold, determining the control strategy as: controlling each automatic guided vehicle to operate according to the current operating mode; when the predicted delay step number is not less than the delay step number threshold, determining the control strategy as: adjusting all or part of the operating modes of each automatic guided vehicle.

[0130] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0131] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for controlling an automatic guided vehicle, characterized in that: include: Obtaining delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, wherein the delay influencing factor is a factor causing the actual running speed of each automated guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; Inputting the state value of the delay influencing factor into a delay step prediction model, so as to process the state value of the delay influencing factor using the delay step prediction model to obtain a predicted delay step number of each of the automatic guided vehicles, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each of the automatic guided vehicles, and the delay step prediction model is a model obtained by reinforcement learning training using multiple sets of training data; The control strategy of each of the automatic guided vehicles is determined according to the predicted delay steps.

2. The automatic guided vehicle control method according to claim 1, characterized in that: Obtain the delay influencing factor state values ​​corresponding to the delay influencing factors of each automated guided vehicle, including: Determining each of the delay influencing factors; Obtaining current status information of each of the automatic guided vehicles; The delay influencing factor state values ​​respectively corresponding to the delay influencing factors of each of the automatic guided vehicles are determined according to the current state information.

3. The automatic guided vehicle control method according to claim 1, characterized in that: Before inputting the delay influencing factor state value into the delay step number prediction model to process the delay influencing factor state value using the delay step number prediction model to obtain the predicted delay step number of each automatic guided vehicle, the method further includes: Acquire the plurality of groups of training data including a plurality of sample delay influencing factor state values ​​and a plurality of sample predicted delay step numbers corresponding to the plurality of sample delay influencing factor state values; The multiple groups of training data are trained to obtain the delay step number prediction model.

4. The automatic guided vehicle control method according to claim 3, characterized in that: Training the multiple sets of training data to obtain the delay step number prediction model includes: Obtaining a weight value of each of the plurality of sample delay influencing factor state values; Generate state data of the delay step number prediction model according to the weight value of each of the plurality of sample delay influencing factor state values ​​and the plurality of sample delay influencing factor state values; Generate action data of the delay step number prediction model according to each of the plurality of sample delay influencing factor state values; Determining a reward value according to the action data, wherein the reward value is used to optimize the weight value; A delay step prediction model is obtained by training according to the state data, the action data and the reward value.

5. The automatic guided vehicle control method according to claim 4, characterized in that: Obtaining a weight value of each of the plurality of sample delay influencing factor state values, including: Determining the degree of delay influence of the sample delay influencing factors corresponding to the state values ​​of the sample delay influencing factors on each of the automatic guided vehicles; The weight value is determined according to the delay impact degree.

6. The automatic guided vehicle control method according to claim 4, characterized in that: Generating state data of the delay step number prediction model according to the weight value of each of the plurality of sample delay influencing factor state values ​​and the plurality of sample delay influencing factor state values, including: Generate a sample state matrix according to the state values ​​of the plurality of sample delay influencing factors; Generate a weight matrix according to the weight values; The state data is determined according to a Hadamard product of the state matrix and the weight matrix.

7. The automatic guided vehicle control method according to claim 4, characterized in that: Generating action data of the delay step number prediction model according to each of the plurality of sample delay influencing factor state values ​​includes: Determine the automated guided vehicle having the fastest average speed in a predetermined time period corresponding to the state values ​​of the plurality of sample delay influencing factors as a reference automated guided vehicle; Obtaining the number of delay steps of the other vehicles in the automated guided vehicles except the reference automated guided vehicle within a delay step prediction period; The delay steps between the automatic guided vehicles are determined as the action data.

8. The automatic guided vehicle control method according to claim 4, characterized in that: Determining a reward value according to the action data includes: The reward value is determined by a reward function, wherein the reward function is: R represents the reward value, represents the actual delay steps of the i-th automatic guided vehicle, A i represents the predicted delay steps of the i-th automated guided vehicle, and n represents the total number of the automated guided vehicles.

9. The automatic guided vehicle control method according to claim 4, characterized in that: The delay step number prediction model is obtained by training according to the state data, the action data and the reward value, including: The weight value is adjusted by using the reward value through a weight value adjustment formula to obtain the adjusted weight value, wherein the weight value adjustment formula is: w ij represents the weight value of the adjusted state value of the j-th sample delay influencing factor of the i-th automated guided vehicle, represents the initial weight value of the state value of the j-th sample delay influencing factor of the i-th automatic guided vehicle, R i represents the reward value of the i-th automated guided vehicle, R k represents the reward value of the k-th automatic guided vehicle; A delay step prediction model is obtained by training according to the state data, the action data and the adjusted reward value.

10. The automatic guided vehicle control method according to claim 4, characterized in that: The delay step number prediction model is obtained by training according to the state data, the action data and the reward value, including: The weight value is adjusted according to the reward value to adjust the weight parameter of the delay step prediction model; Inputting the state data into the delay step number prediction model to obtain output data of the delay step number prediction model; When the difference between the predicted action data corresponding to the output data and the action data is less than a preset difference, the training operation of the delay step prediction model is stopped.

11. The automatic guided vehicle control method according to any one of claims 1 to 10, characterized in that: Determining the control strategy of each of the automatic guided vehicles according to the predicted delay steps includes: When the predicted delay step number is less than the delay step number threshold, determining the control strategy is: controlling each of the automatic guided vehicles to operate according to the current operation mode; When the predicted delay step number is not less than the delay step number threshold, the control strategy is determined as: adjusting the operation mode of all or part of each of the automatic guided vehicles.

12. An automatic guided vehicle control device, characterized in that: include: A first acquisition unit is used to acquire a delay influencing factor state value corresponding to each delay influencing factor of each automated guided vehicle, wherein the delay influencing factor is a factor causing the actual running speed of each automated guided vehicle to be inconsistent with the planned running speed, the planned running speed is a speed pre-planned for each automated guided vehicle by a scheduling system of each automated guided vehicle, and the delay influencing factor state value is a numerical value corresponding to each delay influencing factor; a processing unit, configured to input the delay influencing factor state value into a delay step number prediction model, so as to process the delay influencing factor state value using the delay step number prediction model to obtain a predicted delay step number of each of the automated guided vehicles, wherein the predicted delay step number is used to evaluate the speed difference between the actual running speed and the planned running speed of each of the automated guided vehicles, and the delay step number prediction model is a model trained by reinforcement learning using multiple sets of training data; A determination unit is used to determine the control strategy of each of the automatic guided vehicles according to the predicted delay steps.

13. An automatic guided vehicle, characterized in that: The automatic guided vehicle uses the automatic guided vehicle control method described in any one of claims 1 to 11 above.

14. An automatic guided vehicle control system, characterized in that: The automatic guided vehicle control system uses the automatic guided vehicle control method described in any one of claims 1 to 11 above.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the automatic guided vehicle control method according to any one of claims 1 to 11.

16. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by the processor, the automatic guided vehicle control method according to any one of claims 1 to 11 is executed.

Images