Method and apparatus for simulating an automated guided vehicle (AGV) system

Abstract

A method for simulating an automated guided vehicle (AGV) system includes obtaining sensor data from a virtual AGV in a simulation environment; determining the device status of virtual devices in the simulation environment that interact with the virtual AGV; transmitting the sensor data and device status to the AGV system, receiving AGV motion control information and device operation information from the AGV system; and controlling the motion of the virtual AGV and virtual devices. This method reduces the time and effort spent by deployment engineers on deployment and debugging, reduces factory downtime, and enables verification of the entire AGV system and AGV sensors.

Description

Technical Field

[0001] The present disclosure relates to the field of industrial manufacturing technology, and more particularly, to a method, apparatus, computing device, computer-readable storage medium, and program product for simulating an AGV system. Background Art

[0002] With the development of positioning and navigation technology, automated guided vehicles (AGVs) are increasingly used in industrial environments, such as in factories to move materials between warehouses and production lines or between production lines to improve transportation efficiency. When an AGV performs a material transportation task, the AGV can usually interact with other machines or mechanical equipment running in the factory. The machine or mechanical equipment can be an automated device located on the motion path of the AGV (such as automatic doors, elevators, and signal lights) or other automated equipment related to the transportation task performed by the AGV (such as conveyor belts, robots, lifts, etc.). In addition, when transporting materials in a factory, the AGV may often collide with environmental objects, such as static objects (such as walls or boxes) and dynamic objects (such as workers or other AGVs).

[0003] The interaction between AGVs and other automated equipment requires verification and debugging to achieve smooth material transportation operations and improve factory productivity. Currently, AGV deployment engineers usually verify and debug the interaction between AGVs and other automated equipment on-site based on their experience, and optimize the routing and scheduling algorithms of the dispatch manager.

[0004] Furthermore, in some existing simulators, the AGV controller is virtualized to verify and debug the control algorithm of the AGV, or to separately verify the routing and scheduling algorithms of the scheduling manager. Summary of the Invention

[0005] When AGV deployment engineers verify and debug the interactions between AGVs and other automated equipment, they typically spend considerable time and effort. Furthermore, the AGV deployment process often requires factory shutdowns, which can disrupt normal production and reduce productivity. Furthermore, such debugging relies heavily on the engineers' personal experience and prior preparation, which introduces significant uncertainty into the estimation of factory downtime.

[0006] On the other hand, existing simulators can only verify and debug AGV controllers or routing and scheduling algorithms individually. Simulators cannot verify the interactions between AGVs and other automated equipment, nor can they simultaneously verify the AGV controller, routing and scheduling algorithms, and the interactions between AGVs and other automated equipment. Furthermore, when the AGV controller is fully virtualized, the simulation results in the simulator depend on manually set parameters and are often inconsistent with the actual situation.

[0007] In a first embodiment of the present disclosure, a method for simulating an AGV system is proposed, the method comprising: obtaining sensor data of a virtual AGV in a simulation environment; determining the device status of a virtual device that interacts with the virtual AGV in the simulation environment; sending the sensor data and the device status to the AGV system, and receiving AGV motion control information and device operation information from the AGV system, wherein the AGV motion control information and the device operation information are generated by the AGV system based on the sensor data and the device status; and controlling the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information, respectively.

[0008] In this embodiment, the interaction between the AGV and the automated equipment can be simulated, which can then be verified and debugged. Transport operation times, sequencing, and congestion can also be checked, thereby optimizing routing and scheduling algorithms. Thus, the method of the present disclosure reduces the time and effort spent by deployment engineers on deployment and debugging, and reduces factory downtime. Furthermore, the method of the present disclosure allows verification of the entire AGV system and AGV sensors, not just a portion of the AGV system.

[0009] In a second embodiment of the present disclosure, a device for simulating an AGV system is proposed, which includes: a sensor detection unit, which is configured to obtain sensor data of a virtual AGV in a simulation environment; a state determination unit, which is configured to determine the device state of a virtual device that interacts with the virtual AGV in the simulation environment; a data communication unit, which is configured to send sensor data and device state to the AGV system, and receive AGV motion control information and device operation information from the AGV system, wherein the AGV motion control information and device operation information are generated by the AGV system based on the sensor data and device state; and a motion control unit, which is configured to control the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information, respectively.

[0010] In a third embodiment of the present disclosure, a computing device is provided, which includes: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method in the first embodiment.

[0011] A fourth embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored therein, and the computer-executable instructions are used to execute the method of the first embodiment.

[0012] In a fifth embodiment of the present disclosure, a computer program product is provided, which is tangibly stored in a computer-readable storage medium and includes computer-executable instructions, which, when executed, cause at least one processor to perform the method of the first embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The features, advantages and other aspects of various embodiments of the present disclosure will become more apparent with reference to the accompanying drawings and the following detailed description. Several embodiments of the present disclosure are shown here by way of example and not limitation. In the accompanying drawings,

[0014] Figure 1 A simulation method for an AGV system according to an embodiment of the present disclosure is shown;

[0015] Figure 2 The embodiment according to the present disclosure is shown Figure 1 A simulation system for the simulation method in ;

[0016] Figure 3 (a) to Figure 3 (b) shows Figure 2 Schematic components of a virtual device and a virtual AGV in an embodiment of the present invention;

[0017] Figure 4 (a) to Figure 4 (e) shows Figure 2 A 3D model of an AGV and a plurality of automated devices interacting with the AGV in an embodiment;

[0018] Figure 5 (a) to Figure 5 (d) shows Figure 4 Kinematic models of the AGV and multiple automated devices interacting with the AGV;

[0019] Figure 6 It is displayed through the display interface Figure 4 Schematic diagram of the virtual lidar of the virtual AGV in (a);

[0020] Figure 7 Shows the display interface according to Figure 2 An exemplary simulation environment of an embodiment of the present invention;

[0021] Figure 8 (a) to Figure 8 (b) are displayed through the display interface Figure 4 Schematic diagram of the collision of the virtual AGV with virtual devices and environmental objects in (a);

[0022] Figure 9 The embodiment according to the present disclosure is shown Figure 1 Another simulation system of the simulation method;

[0023] Figure 10 An apparatus for simulating an AGV system according to an embodiment of the present disclosure is shown; and

[0024] Figure 11 is a block diagram of a computing device for simulating an AGV system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although the exemplary methods and apparatus described below include software and / or firmware executed in the hardware of other components, it should be noted that these examples are merely illustrative and should not be considered restrictive. For example, it is contemplated that any or all of the hardware, software, and firmware components may be implemented in dedicated hardware, in dedicated software, or in any combination of hardware and software. Therefore, although exemplary methods and apparatus have been described below, those skilled in the art will readily appreciate that the examples provided are not intended to limit the manner in which the methods and apparatus are implemented.

[0026] In addition, the possible implementation architectures, functions and operations of the methods and systems according to various embodiments of the present disclosure are shown in the flowcharts and block diagrams of the accompanying drawings. It should be noted that the functions marked in the boxes may also appear in an order different from the order marked in the drawings. For example, depending on the functions involved, two consecutive blocks may be executed substantially in parallel, or they may sometimes be executed in the opposite order. It should also be noted that each box in the block diagrams and / or flow charts and the combination of boxes in the block diagrams and / or flow charts may be implemented using a dedicated hardware-based system for performing the specified function or action, or may be implemented using a combination of dedicated hardware and computer instructions.

[0027] As used herein, the terms "including," "comprising," and similar terms are open-ended terms, meaning "including, but not limited to," indicating that additional content may also be included. The term "based on" means "based, at least in part, on." The term "an embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and so on.

[0028] Figure 1 A method for simulating an automated guided vehicle (AGV) system according to an embodiment of the present disclosure is shown. The AGV system is an actual system in a factory and includes at least a scheduling manager and an AGV controller. The scheduling manager schedules and routes AGVs based on AGV tasks, AGV states received from various AGV controllers, and corresponding device states received by corresponding automated devices that interact with the AGV when performing tasks. The scheduling manager generates AGV control information and device operation information for controlling the movement of the AGV and the automated devices that interact with the AGV, respectively. The AGV controller controls the movement of the motor of the AGV based on the AGV control information and the current position and current orientation of the AGV.

[0029] Reference Figure 1 , method 100 starts from step 101. In step 101, sensor data of a virtual AGV in a simulation environment is obtained. A simulation environment consistent with the actual factory environment is established, and the material transportation scenario in the factory is simulated. The simulation environment includes a virtual AGV, virtual equipment related to the transportation task performed by the virtual AGV (e.g., conveyor belts, robots, elevators, etc.), virtual equipment located on the motion path of the virtual AGV (e.g., automatic doors, elevators, signal lights, etc.), and environmental objects around the virtual AGV (e.g., walls, boxes, workers, other AGVs, etc.). The sensor data of the virtual AGV in the simulation environment can be used to calculate the position and orientation of the virtual AGV in the simulation environment. In some embodiments, sensor data can be periodically obtained to periodically calculate the position and orientation of the virtual AGV in the simulation environment.

[0030] In some embodiments, step 101 of obtaining sensor data of a virtual AGV in a simulation environment further includes ( Figure 1(not shown): sensor data is obtained based on the sensor model of the virtual AGV, and the sensor model of the virtual AGV includes the position and parameters of the virtual sensor of the virtual AGV. The virtual AGV includes three models: a 3D model, a kinematic model, and a sensor model. The 3D model describes the outline of the AGV body, that is, the shape and size. The kinematic model describes the motion relationship and motion parameters of the moving parts of the AGV body, such as the connection relationship, motion type, motion direction, and motion limit of the moving parts. The sensor model describes the position and sensor parameters of the AGV sensor relative to the AGV body. The AGV sensor can be a lidar, a safety light curtain, a trajectory tracker, a camera, etc., so the sensor parameters can vary according to different types of sensors. For example, for a lidar, the sensor parameters may include a scanning beam (e.g., 12 / 24 / 48 beams), a scanning angle (e.g., 120° / 180° / 360°), and an effective distance of the beam (e.g., 1 / 2 / 5m). In order to obtain the sensor data of the virtual AGV, similar to the virtual AGV, the virtual device and / or the environmental objects around the virtual AGV also include a 3D model. If the virtual device and / or environment objects are movable, a kinematic model is also included.

[0031] Therefore, sensor data can be obtained based on the position of the virtual sensor and the parameters of the virtual sensor defined in the sensor model of the virtual AGV. When a virtual device and / or environmental object exists within the detection range of the virtual sensor (for example, within the scanning angle range of the lidar or within the field of view of the camera), the external contour data defined by the 3D model of the virtual device and / or environmental object can be further used to obtain sensor data. In addition, the sensor data obtained can vary according to the different sensor types represented by the sensor model. For example, for a lidar, the sensor data can be a data set corresponding to the number of beams of the lidar and having a value less than or equal to the effective distance of the beam. For another example, for a safety light curtain, the sensor data can be a value of 0 or 1, and for a camera, the sensor data can be an RGB image or an RGB-D image.

[0032] In the above manner, AGV sensors and other virtual devices and / or environmental objects in an actual factory are simulated in a simulation environment, so that the obtained sensor data is close to the sensor data obtained by the AGV in the actual factory.

[0033] Still refer to Figure 1In step 102, the device states of virtual devices interacting with the virtual AGV in the simulation environment are determined. Virtual devices interacting with the virtual AGV can include virtual devices related to the transportation task performed by the virtual AGV (e.g., conveyor belts, robots, elevators, etc.) and virtual devices located on the virtual AGV's motion path (e.g., automatic doors, elevators, traffic lights, etc.). Device states can vary depending on the type of virtual device. For example, for an automatic door, the device state can include open, opening, closed, closing, or waiting. For another example, for an elevator, the device state can include the floor the elevator is located on and whether the elevator door is open, opening, closed, closing, or waiting. For a conveyor belt, the device state can include running and non-running states. The device states are provided to the scheduling manager, allowing the scheduling manager to control the movement of the virtual devices and virtual AGV, respectively, to implement interaction between the virtual AGV and the virtual devices. It should be noted that step 102 can be performed sequentially or synchronously with step 101, or in more cases, independently of step 101.

[0034] In some embodiments, step 102 of determining the device state of a virtual device interacting with a virtual AGV in a simulation environment further includes: obtaining state data of the virtual device based on a sensor model of the virtual device, the state data representing the relative motion between the moving components of the virtual device, wherein the sensor model of the virtual device includes the positions and parameters of virtual sensors of the virtual device; and determining the device state of the virtual device based on the state data. In an embodiment, similar to the virtual AGV, three models are constructed for the virtual device: a 3D model, a kinematic model, and a sensor model. The 3D model describes the outline of the virtual device, i.e., its shape and size. The kinematic model describes the motion relationships and motion parameters of the moving components of the virtual device, such as the connection relationships, motion types, motion directions, and motion limits of the moving components. The sensor model describes the position (e.g., at the joints of the moving components) and parameters of the virtual sensor, which is used to detect the relative motion between the moving components. For example, for an automatic door, the virtual sensor can be a virtual lidar at the joint between the door frame and the door leaf, and the virtual sensor parameters can include a scanning beam, a scanning angle, and a beam effective range. Alternatively, the virtual sensor can be a virtual position sensor located at the junction between the door frame and the door leaf (sensing the movement of the door panel position), and the relative movement of the door frame and the door leaf is detected by the change in the coordinate position of the door leaf. For another example, for an elevator, the virtual sensor can be a virtual height sensor on the car or on the ground and a virtual lidar or virtual position sensor located at the junction between the door frame and the door leaf of the car. The state data obtained can vary according to the different sensor types represented by the sensor model of the virtual device. For example, for an automatic door, the state data can be data representing the degree of opening of the automatic door (for example, 0 to 100), while for an elevator, the state data can be data representing the floor number and degree of opening of the car. After obtaining the state data of the virtual device, the device state of the virtual device is determined based on the state data. As mentioned above, the device state may vary according to different virtual devices. The state data can be mapped to different device states according to preset rules.

[0035] In this way, the automation equipment in the actual factory that interacts with the AGV during transportation tasks is simulated in the simulation environment, and the device status of the virtual automation equipment can be obtained, so that the virtual AGV and the virtual equipment interacting with the virtual AGV can be controlled separately. Therefore, the interaction between the AGV and the automation equipment can be simulated almost realistically.

[0036] Still refer to Figure 1In step 103, the sensor data and device status are sent to the AGV system, and AGV motion control information and device operation information are received from the AGV system, wherein the AGV motion control information and device operation information are generated by the AGV system based on the sensor data and device status. The AGV system is an actual system outside the simulation environment. According to the kinematic model of the AGV, the AGV motion control information may be motor control information or position control information. The motor control information may include the rotation direction and rotation angle of the motor of the virtual AGV, and the position control information may include the overall motion direction, rotation direction and rotation angle of the virtual AGV and the travel speed of the virtual AGV. The device operation information may include the target position and execution action of the virtual device, such as the opening / closing of the automatic door, the floor to be reached by the elevator and the opening / closing of the elevator, the rising / falling of the elevator, etc.

[0037] In some embodiments, the AGV system includes an AGV controller and a dispatch manager. Sensor data from the virtual AGV is sent to the AGV controller, and device status of the virtual device is sent to the dispatch manager. Different communication interfaces can be established based on the interface definitions of the AGV controller (e.g., TCP / IP, OPC, Profinet, etc.) and the dispatch manager (e.g., TCP / IP, OPC, Profinet, etc.).

[0038] In some embodiments, the sensor data of the virtual AGV is periodically sent to the AGV controller, and in response to a request from the scheduling manager, the device status of the virtual device is sent to the scheduling manager. Specifically, after the AGV controller receives the sensor data of the virtual AGV, the current position and current orientation of the virtual AGV in the simulation environment can be calculated based on the local map data and sensor data of the AGV controller, that is, the virtual AGV is positioned. Next, the AGV controller, in response to a request from the scheduling manager, periodically sends the current position and current orientation of the virtual AGV, the transportation task performed by the virtual AGV, the target position and speed parameters (e.g., acceleration, maximum speed, motor speed, etc.), the power of the virtual AGV and the elapsed running time, and other information as the status data of the AGV to the scheduling manager.

[0039] The scheduling manager determines the virtual device to be operated based on the AGV status data from the AGV controller and the route map of the transportation task performed by the virtual AGV, and sends a device status request to the simulator. The simulator sends the device status of the virtual device to the scheduling manager in response to the request. Then, the scheduling manager generates AGV motion control information and device operation information based on the AGV status data and device status from the AGV controller. The AGV motion control information is used to enable the AGV to reach the desired position and perform the desired action, and the device operation information is used to enable the virtual device to reach the desired position and perform the desired action. The scheduling manager continues to send device status requests to the simulator. When the received device status of the virtual device is in the desired state, the scheduling manager again sends AGV control information to the AGV controller to enable the virtual AGV in the simulation environment to interact with the virtual device.

[0040] After receiving the AGV control information, the AGV controller generates motor control information of the virtual AGV based on the AGV control information and the current position and current orientation of the virtual AGV, or further converts the motor control information into position control information, and sends the position control information to the simulator.

[0041] Next, method 100 proceeds to step 104, in which the motion of the virtual AGV and virtual device in the simulation environment is controlled based on the AGV motion control information and the device operation information, respectively. After receiving the AGV motion control information and the device operation information, the simulator controls the virtual AGV to move to the target position and perform the corresponding action based on the AGV target position and execution action included in the AGV motion control information, and controls the virtual device to move to the target position and perform the corresponding action based on the execution action included in the device operation information. By controlling the motion and action of the virtual AGV and virtual device, respectively, the virtual AGV can interact with the virtual device in the simulation environment. For example, an elevator unloads materials to the virtual AGV, and the virtual AGV passes through an automatic door and rides an elevator to a conveyor belt to unload materials, etc.

[0042] In some embodiments, controlling the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information further includes: Figure 1 (not shown): determining the next position of the moving parts of the virtual AGV based on the AGV motion control information, the kinematic model of the virtual AGV, and the current position of the moving parts of the virtual AGV; and determining the next position of the moving parts of the virtual device based on the device operation information, the kinematic model of the virtual device, and the current position of the moving parts of the virtual device, wherein the kinematic models of the virtual AGV and the virtual device include the motion relationship and motion parameters of the virtual AGV and the moving parts of the virtual device.

[0043] As mentioned above, the kinematic model of the AGV and the kinematic model of the virtual device define the motion relationship and motion parameters of the mobile components of the AGV and the virtual device, respectively. The AGV motion control information may be motor control information or position control information. The motor control information may include the rotation direction and rotation angle of the motor of the virtual AGV, and the position control information may include the overall motion direction, rotation direction, and rotation angle of the virtual AGV, as well as the travel speed of the virtual AGV. The next position of the mobile components of the AGV is calculated based on the AGV motion control information, the kinematic model of the AGV, and the current position of the mobile components of the AGV. The device operation information includes the target position and the execution action of the virtual device. The device operation information is converted into the target position of the virtual device and the control information for the connection between the mobile components of the virtual device. The next position of the mobile components of the virtual device is calculated based on the target position of the virtual device and the converted control information for the connection, the kinematic model of the virtual device, and the current position of the mobile components of the virtual device. The calculated next positions of the mobile components of the virtual AGV and the mobile components of the virtual device can be provided to a collision detection unit (described below) for collision detection and a 3D rendering unit for displaying a 3D image.

[0044] The motion of the virtual AGV and virtual equipment in the simulation environment is controlled based on the AGV motion control information and equipment operation information from the actual AGV system, respectively, so that the virtual AGV and virtual equipment in the simulation environment can be controlled by actual control signals without relying on manual input, and therefore the simulation results are closer to the situation in the actual factory.

[0045] In some embodiments, method 100 further includes ( Figure 1 (not shown): Determine whether the virtual AGV has collided with or come too close to virtual equipment and / or environmental objects surrounding the virtual AGV. In a real factory, an AGV might collide with virtual equipment, static or dynamic objects around the AGV, or other AGVs while performing a transport task, indicating a problem with the AGV deployment. Therefore, collision detection can be performed in a simulation environment.

[0046] In some embodiments, determining whether the virtual AGV has collided with or is too close to the virtual device and / or environmental objects surrounding the virtual AGV further includes: determining a distance between the virtual AGV and the virtual device and / or environmental objects surrounding the virtual AGV based on a 3D model of the virtual AGV and a 3D model of the environmental objects surrounding the virtual AGV; and determining whether the virtual AGV has collided with or is too close to the virtual device and / or environmental objects surrounding the virtual AGV based on the determined distance, wherein the 3D model of the virtual AGV and the 3D model of the environmental objects surrounding the virtual device and / or the virtual AGV respectively include outline data of the virtual AGV and the environmental objects surrounding the virtual device and / or the virtual AGV. The distance between the AGV and the virtual device and / or environmental objects can be obtained by performing a calculation using the outline data defined by the 3D model of the virtual AGV, the outline data defined by the 3D model of the environmental objects surrounding the virtual device and / or the virtual AGV, and the current position of the environmental objects surrounding the virtual device and / or the virtual AGV. When the distance is less than or equal to zero, this indicates that the AGV has intruded into the virtual device and / or environmental objects. When the distance is less than the threshold, this indicates that the AGV is too close to the virtual device and / or environmental objects.

[0047] By detecting collisions between the virtual AGV and virtual equipment and / or environmental objects surrounding the virtual AGV, collision events in the actual factory environment can be simulated to help the simulator user adjust the deployment of the AGV.

[0048] In some embodiments, method 100 further includes ( Figure 1 (not shown): Based on the 3D models of all virtual AGVs, virtual devices and environmental objects in the simulation environment, 3D images of all virtual AGVs, virtual devices and environmental objects are displayed through the display interface. By using the 3D models and current positions of all virtual AGVs, virtual devices and environmental objects, the 3D images of all virtual AGVs, virtual devices and environmental objects can be displayed on the display interface through 3D rendering technology, so that the movement and / or operation of AGVs and automation equipment similar to the movement and / or operation of AGVs and automation equipment in the actual factory can be observed on the display interface. In addition to the 3D models, the results of collision detection can also be displayed on the display interface to highlight the collision and / or excessive proximity of the AGV with the virtual devices and / or environmental objects around the AGV.

[0049] Visualization effects are achieved by displaying 3D images of virtual AGVs, virtual devices, and environmental objects, which can help simulator users directly observe the interaction process between virtual AGVs and virtual devices and the entire simulation, making the debugging and deployment of AGVs easier.

[0050] Virtual sensor data can be generated in a simulation environment by simulating the sensors of the AGV in a simulator, so that the actual AGV controller obtains sensor data from the virtual world rather than the real world. In addition, virtual device states can be generated in a simulation environment by simulating the automation equipment in a simulator, so that the scheduling manager controls the virtual automation equipment and virtual AGV in the simulation environment, thereby simulating the interaction between the AGV and the automation equipment, and then verifying and debugging the interaction. After the simulation runs, by viewing the AGV task execution time, AGV waiting time, and other indicators of the scheduling manager, the transportation operation time, sequence, and congestion can also be checked, and then the routing and scheduling algorithms can be optimized. Therefore, according to the method of the present disclosure, through virtual simulation and debugging, the time and effort spent by deployment engineers in deployment and debugging are reduced, and factory downtime is reduced. In addition, using the method of the present disclosure, the entire AGV system (including the AGV controller and scheduling manager) and AGV sensors can be verified, not just part of the AGV system.

[0051] The following describes a method for simulating an AGV system with reference to specific implementations.

[0052] Figure 2 The embodiment according to the present disclosure is shown Figure 1 The simulation system of the simulation method in Figure 2 In the simulation system 200, a simulator 21, an AGV system 22, and a command generator 23 are both actual systems in a factory. The command generator 22 can be an external system running in the factory, such as a warehouse management system (WMS), a manufacturing execution system (MES), or other system that can send transport commands to the AGV system. The contents of the transport command may include, for example, the transport starting point, the transport destination, the type and quantity of materials, time requirements, etc. The AGV system 22 includes an adapter 221, a scheduling manager 222, and multiple AGV controllers. The adapter 221 receives the transport command from the command generator 22 and converts the transport command into multiple AGV transport tasks. The AGV transport tasks are then sent to the scheduling manager 222 based on the material type and maximum carrying capacity of the different AGVs. Each AGV transport task may include a transport starting point, a transport destination, and the required AGV number. In addition, the adapter 221 can prioritize the AGV transport tasks based on the time requirements in the transport command.

[0053] In an actual factory, the scheduling manager 222 schedules and routes all AGVs controlled by multiple AGV controllers 223 based on the AGV transport tasks from the adapter 221, the AGV status data from multiple AGV controllers 223, and the device status from the device server, based on preset routing and scheduling rules. The scheduling manager 222 sends device control information to the device server and sends AGV control information to the AGV controller 223 to control the automation equipment and AGVs accordingly, so that the transport tasks are assigned to the AGVs, and the AGVs interact with the relevant automation equipment during the AGVs' execution of the transport tasks. The device server can be a warehouse control system (WCS), a supervisory control and data acquisition system (SCADA), a distributed control system (DCS), or other systems that control the automation equipment in the factory. In the simulation system 200, the automation equipment is virtualized in the simulator 21 without using any actual automation equipment. Therefore, the device status received by the scheduling manager 222 comes from the virtual equipment in the simulator 21, and the generated device control information is also sent to the simulator 21 to control the virtual equipment.

[0054] In an actual factory, the AGV controller 222 calculates the current position and current orientation of the AGV controlled by the AGV controller based on the received sensor data (e.g., scanning data from a lidar, image data from a camera, response results from a trajectory tracker, etc.), that is, locates the AGV controlled by the AGV controller. The AGV controller 223 also sends the current position and current orientation of the AGV obtained by calculation, the transportation task performed by the AGV, the target position and speed parameters (e.g., acceleration, maximum speed, motor speed, etc.), the power of the AGV and the elapsed running time, and other information as the status data of the AGV to the scheduling manager. In this embodiment, in response to a request from the scheduling manager 222, the AGV controller 223 periodically sends the status data of the AGV to the scheduling manager 222, so that the scheduling manager 222 can obtain the status data of the AGV in real time. The AGV controller 223 also generates motor control information based on the AGV control information received from the scheduling manager 222 and the current position and current orientation of the AGV to control the movement of the AGV. In simulation system 200, the AGV is virtualized in simulator 21 without using any actual AGV. Therefore, the sensor data received by AGV controller 223 comes from the virtual AGV in simulator 21, and motor control information is sent to simulator 21 to control the virtual AGV. When the kinematic model of the virtual AGV does not include motors, the AGV controller can further convert the motor control information into position control information and send the position control information to the virtual AGV 212. In this embodiment, AGV controller 223 can be an actual hardware controller or a simulated controller running on a PC or embedded system.

[0055] Reference Figure 2 and Figure 3 , the simulator 21 includes several virtual devices (e.g., virtual devices 210 and 210′), a virtual device server 211, several virtual AGVs (e.g., virtual AGVs 212 and 212′), a device signal transmission unit 213, an AGV connection unit 214, a motion control unit 215, a collision detection unit 216, and a 3D rendering unit 217. The number and type of virtual devices and virtual AGVs in the simulator 21 can vary depending on different factory environments. It should be noted that although Figure 2 Although not shown, the simulator 21 also includes 3D models and / or kinematic models of other environmental objects, such as static objects (e.g., walls and boxes) and dynamic objects (e.g., workers). The following description is based on a virtual device 210 and a virtual AGV 212. It should be noted that the other virtual devices and virtual AGVs have the same or similar components as the virtual device 210 and virtual AGV 212.

[0056] like Figure 3 As shown in (a), the virtual device 210 includes a 3D model 2100, a kinematic model 2101, a sensor model 2102, and a state determination unit 2103. The 3D model 2100 describes the outline of the virtual device, i.e., its shape and size. The kinematic model 2101 describes the motion relationships and motion parameters of the moving components of the virtual device, such as the connection relationships, motion types, motion directions, and motion limits of the moving components. The sensor model 2102 is used to detect the relative motion between the moving components and may include the location (e.g., at the joints of the moving components) and parameters of the virtual sensors. The state determination unit 2103 is configured to obtain state data of the virtual device based on the sensor model 2102 and determine the device state of the virtual device based on the state data. The state determination unit 2103 periodically determines the device state of the virtual device 210 and stores the device state in the virtual device server 211. In addition, the state determination unit 2103 converts device operation information from the scheduling manager 222 into a target position of the virtual device and joint control information for the joints between the moving components of the virtual device.

[0057] The device signal transmission unit 213 is configured to implement communication between the scheduling manager 222 and the virtual device 210, which can be defined based on the actual interface of the scheduling manager 222. In this embodiment, the device signal transmission unit 213 transmits the current device status of the corresponding virtual device on the device server 211 to the scheduling manager 222 in response to a request from the scheduling manager 222. In addition, the device signal transmission unit 213 also receives device operation information from the scheduling manager 222 and transmits the device operation information to the status determination unit 2103.

[0058] like Figure 3 As shown in (b), the virtual AGV 212 includes a 3D model 2120, a kinematic model 2121, a sensor model 2122, and a sensor detection unit 2123. The 3D model 2120 describes the outline of the AGV body, i.e., the shape and size. The kinematic model 2121 describes the motion relationship and motion parameters of the moving parts of the AGV body, such as the connection relationship, motion type, motion direction, and motion limit of the moving parts. The sensor model 2123 describes the position and sensor parameters of the AGV sensor relative to the AGV body. The sensor detection unit 2123 obtains sensor data based on the sensor model of the virtual AGV and the 3D model of the virtual device and / or the environmental objects surrounding the virtual AGV.

[0059] The AGV connection unit 214 is configured for communication between the AGV controller 223 and the virtual AGV 212, which can be defined according to the actual interface of the AGV controller 223. In this embodiment, the AGV connection unit 214 periodically sends sensor data obtained by the virtual AGV 212 to the AGV controller, so that the AGV controller 223 can locate the virtual AGV 212 in real time.

[0060] The motion control unit 215 receives motor control information or position control information and device operation information via the device signal transmission unit 213 and the AGV connection unit 214, respectively. The motion control unit 215 calculates the next position of the moving parts of the virtual AGV 212 based on the motor control information or position control information, the motion relationships and motion parameters defined in the kinematic model 2121 of the virtual AGV, and the current position of the moving parts of the virtual AGV 212. Furthermore, the motion control unit 215 determines the next position of the moving parts of the virtual device 210 based on the device operation information, the kinematic model 2101 of the virtual device 210, and the current position of the moving parts of the virtual device 210. The motion control unit 215 provides the calculated next positions of the moving parts of the virtual AGV 212 and virtual device 210 to the collision detection unit 216 for collision detection. The 3D rendering unit 217 is configured to display the motion of the virtual AGV 212 and virtual device 210 through a display interface.

[0061] The collision detection unit 216 detects collisions based on the 3D model 2120 of the virtual AGV and the environmental objects ( Figure 2 The distance between the virtual AGV and the virtual device and / or the environmental objects surrounding the virtual AGV is calculated based on the contour data defined by the 3D model of the virtual AGV (not shown) and the position of the virtual AGV, virtual device and / or environmental objects. When the distance is less than or equal to zero, it indicates that the virtual AGV 210 has collided with the virtual device and / or the environmental objects surrounding the virtual AGV. When the distance is less than a threshold, it indicates that the virtual AGV 210 is too close to the virtual device and / or the environmental objects surrounding the virtual AGV. Afterwards, the collision detection unit 216 displays a detection report through a display interface for the user to view.

[0062] The 3D rendering unit 217 displays 3D images of all virtual AGVs, virtual devices, and environmental objects in the simulation environment based on their 3D models. The 3D rendering unit 217 can display the entire simulation environment and the entire motion process of all virtual AGVs and virtual devices during the simulation on the display interface.

[0063] In this embodiment, the interaction between AGVs and automated equipment can be simulated, which can then be verified and debugged. Transport operation times, sequencing, and congestion can also be checked, thereby optimizing routing and scheduling algorithms. Consequently, according to this embodiment, the time and effort spent by deployment engineers on deployment and debugging, as well as factory downtime, are reduced. Furthermore, according to this embodiment, the entire AGV system and AGV sensors can be verified, not just a portion of the AGV system.

[0064] Figure 4 (a) to Figure 4 (e) shows Figure 2 A 3D model of an AGV and automated equipment interacting with the AGV in an embodiment. Figure 5 (a) to Figure 5 (d) shows Figure 4 Kinematic model of the AGV and the automated equipment interacting with the AGV. Figure 6 It is displayed through the display interface Figure 4 Schematic diagram of the virtual lidar of the virtual AGV in (a). Figure 7 Shows the display interface according to Figure 2 Schematic simulation environment of the embodiment of the invention. Figures 4 to 7 In the embodiment of the present invention, the dispatch manager 222 dispatches an AGV to load materials from an elevator on the second floor and transfer the materials to a conveyor belt on the first floor. During the transport path, the AGV passes through an automatic door and takes an elevator to the first floor. Therefore, the automated equipment that interacts with the AGV includes elevators, automatic doors, elevators, and conveyor belts.

[0065] like Figure 4 (a) to Figure 4 As shown in (e), the shapes and sizes of AGV, lift, automatic door, elevator and conveyor belt are defined in the 3D model. Figure 5 In the kinematic model of (a), the various moving parts of the AGV, namely the base and the lifting platform, are shown. The motion type of the two moving parts, the base and the lifting platform, is limited to parallel motion, the motion direction is limited to upward and downward, and the maximum motion range and other motion parameters are limited. Similarly, in Figure 5 In the kinematic model of (b), the various moving parts of the lift are shown, namely the base and the lifting platform; Figure 5 In the kinematic model of (c), the moving parts of the automatic door, namely the door frame and the door leaf, are shown; Figure 5 The kinematic model in (d) shows the moving parts of the elevator: the elevator shaft, the car body, the two car doors, the two doors on the second floor, and the two doors on the first floor. Since the conveyor belt moves as a whole, a kinematic model cannot be built for it.

[0066] In this embodiment, a lidar is used as a sensor for the AGV. Therefore, the sensor model of the virtual AGV includes the position of the lidar relative to the AGV body and the scanning beam of the lidar (e.g., 12 / 24 / 48 beams), the scanning angle (e.g., 120° / 180° / 360°), and the effective distance of the beam (e.g., 1 / 2 / 5m). In this way, when the virtual AGV obtains sensor data in the simulation environment, the sensor data can be obtained based on the position of the lidar and the parameters of the lidar defined in the sensor model. In this embodiment, the sensor data is a data set corresponding to the number of beams of the lidar and having a value less than or equal to the effective distance of the beam, for example, [v1, v2, v3…v i ] where i is the beam of the lidar, and v is the distance between the position of the lidar and the virtual device and / or environmental object for each beam (when the virtual device and / or environmental object exists within the effective range of the beam) or the effective range of the beam (when the virtual device and / or environmental object does not exist within the effective range of the beam). Figure 6 It is displayed through the display interface Figure 4 Schematic diagram of the virtual lidar of the virtual AGV in (a).

[0067] In this embodiment, a sensor model is established for each of the elevator, automatic door, and elevator. Virtual sensors are placed at the joints of moving parts to detect relative motion between them. The elevator's virtual sensor is located at the joint between the base and the lifting platform, and its relative motion can be detected by changes in the platform's coordinate position. Similarly, the automatic door's virtual sensor is located at the joint between the door frame and the door leaf. Elevators have two virtual sensors: a virtual height sensor on the car or ground, and a position sensor at the joint between the car's door frame and the door leaf.

[0068] In this embodiment, when simulating the AGV system 22, the virtual AGV periodically obtains sensor data based on the lidar model and sends the sensor data to the AGV controller 223. The AGV controller 223 locates the current position of the virtual AGV and, in response to a request from the dispatch manager 222, periodically sends the virtual AGV's status data to the dispatch manager 222. The virtual elevator, virtual automatic door, virtual elevator, and virtual conveyor belt periodically determine the current status of the virtual devices and save this status in the virtual device server 211. The dispatch manager 222 determines the virtual device with which the virtual AGV will interact based on the virtual AGV's current position and sends a device status request for the virtual device to the virtual device server 211. Based on the current state of the virtual device returned by the virtual device server 211, the scheduling manager 222 generates device operation information and sends the device operation information to the virtual device to control the virtual device to reach the desired device state (for example, so that the virtual elevator can reach the destination floor and open), and generates AGV control information and sends the AGV control information to the AGV controller 223 to control the action of the virtual AGV (for example, waiting at an appropriate location). When the virtual device that will interact with the virtual AGV is in the desired device state, the scheduling manager 222 sends the AGV control information to the AGV controller 223 so that the AGV controller 223 can control the movement of the virtual AGV (for example, entering the elevator).

[0069] Meanwhile, during the movement of the virtual AGV, the collision detection unit 216 determines whether the virtual AGV collides with or comes too close to the virtual device and / or the environmental objects surrounding the virtual AGV, and displays the collision detection result through the display interface. The collision detection result includes time, moving part name 1, moving part name 2, distance, and status (collision / too close). Figure 7 Shows the display interface according to Figure 2 Schematic simulation environment of the embodiment of the present invention. Figure 7 As shown, the AGV system and the interaction between AGV and automation equipment can be simulated to be close to the actual factory. Figure 8 (a) to Figure 8 (b) are displayed through the display interface respectively Figure 4 Schematic diagram of the collision of the virtual AGV with virtual equipment and environmental objects in (a).

[0070] Figure 9 The embodiment according to the present disclosure is shown Figure 1Another simulation system of the simulation method in . The simulation system 900 includes a main simulator 21 and a plurality of auxiliary simulators 91. The main simulator 21 and the auxiliary simulators 91 can run in different hardware devices to increase the simulation speed. In this embodiment, the 3D model 2120 or 2120′ of each AGV is retained in the main simulator 21, and the kinematic model (e.g., kinematic model 2121), the sensor model (e.g., the sensor detection model 2122) and the sensor detection unit (e.g., the sensor detection unit 2123) of each AGV are deployed in the auxiliary simulator 91. The number of auxiliary simulators can be determined as needed. For example, one auxiliary simulator 91 can be arranged for each AGV, or one auxiliary simulator 91 can be arranged for multiple AGVs.

[0071] In the auxiliary simulators 91, in addition to the AGV's kinematic model, sensor model, and sensor detection unit, each auxiliary simulator 91 also includes a motion control unit 215' and an AGV connection unit 214' dedicated to the virtual AGV. The motion control unit 215' is configured to calculate the next position of the AGV's moving parts based on the AGV control information, the virtual AGV's kinematic model 2121, and the current position of the AGV's moving parts. The AGV connection unit 214' is configured to enable communication between the virtual AGV in the auxiliary simulator 91 and the AGV controller 223. Figure 9 Other components or units in Figure 2 The components or units in FIG. 1 are the same, and their description is omitted in this article.

[0072] During simulation, the sensor detection unit 2123 in the auxiliary simulator 91 sends the sensor data of the virtual AGV to the AGV controller 223 via the AGV connection unit 214′, and the AGV controller 223 positions the virtual AGV and sends AGV motion control information to the motion control unit 215′ via the AGV connection unit 214′. The motion control unit 215 determines the next position of the AGV's moving parts and sends the next position to the AGV controller 223 via the AGV connection unit 214′. The AGV controller 223 again sends the next position of the moving parts to the main simulator 21 via the AGV connection unit 214, so that collision detection is performed by the collision detection unit 216 and the 3D rendering unit 217 and a 3D image is displayed.

[0073] Figure 10 FIG. 4 shows an apparatus for simulating an AGV system according to an embodiment of the present disclosure. Figure 10, the device 1000 includes a sensor detection unit 1001, a state determination unit 1002, a data communication unit 1003 and a motion control unit 1004. The sensor detection unit 1001 is configured to obtain sensor data of a virtual AGV in a simulation environment. The state determination unit 1002 is configured to determine the device state of a virtual device that interacts with the virtual AGV in the simulation environment. The data communication unit 1003 is configured to send the sensor data and the device state to the AGV system, and receive AGV motion control information and device operation information from the AGV system, wherein the AGV motion control information and the device operation information are generated by the AGV system based on the sensor data and the device state. The motion control unit 1004 is configured to control the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information, respectively. Figure 9 The units in can be implemented by software, hardware (eg, integrated circuit, FPGA, etc.) or a combination of software and hardware.

[0074] In some embodiments, the sensor detection unit 1001 is further configured to: acquire sensor data based on a sensor model of the virtual AGV, where the sensor model of the virtual AGV includes positions and parameters of virtual sensors of the virtual AGV.

[0075] In some embodiments, the state determination unit 1002 is further configured to: obtain state data of the virtual device based on a sensor model of the virtual device, the state data representing the relative movement between the moving parts of the virtual device, wherein the sensor model of the virtual device includes the position and parameters of the virtual sensor of the virtual device; and determine the device state of the virtual device based on the state data.

[0076] In some embodiments, the motion control unit 1004 is further configured to: determine the next position of the moving parts of the virtual AGV based on the AGV motion control information, the kinematic model of the virtual AGV, and the current position of the moving parts of the virtual AGV; and determine the next position of the moving parts of the virtual device based on the device operation information, the kinematic model of the virtual device, and the current position of the moving parts of the virtual device, wherein the kinematic models of the virtual AGV and the virtual device include the motion relationship and motion parameters of the virtual AGV and the moving parts of the virtual device.

[0077] In some embodiments, the apparatus 1000 further includes a collision detection unit ( Figure 9 (not shown in the figure), the collision detection unit is configured to determine whether the virtual AGV collides with or is too close to the virtual device and / or environmental objects around the virtual AGV.

[0078] In some embodiments, the collision detection unit is further configured to: determine the distance between the virtual AGV and the virtual device and / or the environmental objects surrounding the virtual AGV based on the 3D model of the virtual AGV and the 3D models of the environmental objects surrounding the virtual device and / or the virtual AGV; and determine whether the virtual AGV collides with or is too close to the virtual device and / or the environmental objects surrounding the virtual AGV based on the determined distance, wherein the 3D model of the virtual AGV and the 3D models of the environmental objects surrounding the virtual device and / or the virtual AGV respectively include contour data of the virtual AGV and the environmental objects surrounding the virtual device and / or the virtual AGV.

[0079] In some embodiments, the apparatus 1000 further includes a 3D rendering unit ( Figure 9 (not shown), the 3D rendering unit is configured to display 3D images of all virtual AGVs, virtual devices and environmental objects through a display interface based on the 3D models of all virtual AGVs, virtual devices and environmental objects in the simulation environment.

[0080] Figure 11 is a block diagram of a computing device for simulating an AGV system according to an embodiment of the present disclosure. Figure 11 It can be seen that the computing device 1100 for simulating the AGV system includes a processor 1101 and a memory 1102 coupled to the processor 1101. The memory 1102 is configured to store computer-executable instructions, which, when executed, enable the processor 1101 to perform the method in the above embodiment.

[0081] In addition, alternatively, the method can be implemented by a computer-readable storage medium. The computer-readable storage medium stores computer-executable program instructions for executing the method according to various embodiments of the present disclosure. The computer-readable storage medium can be an actual device that can retain and store the instructions used by the instruction execution device. The computer-readable storage medium can be, for example, but not limited to: an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. In a more specific example (non-exhaustive list), the computer-readable storage medium includes a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or a raised structure in a groove in which instructions are stored) and any suitable combination thereof. Computer-readable storage media as used herein is not to be construed as transient signals per se, such as freely propagating radio waves or other electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses propagating through fiber optic cables), or electrical signals transmitted through wires.

[0082] Therefore, in another embodiment, a computer-readable storage medium is provided in the present disclosure, wherein the computer-readable storage medium stores computer-executable instructions for executing the method according to various embodiments of the present disclosure.

[0083] In another embodiment, a computer program product is provided in the present disclosure. The computer program product is tangibly stored in a computer-readable storage medium and includes computer-executable instructions that, when executed, cause at least one processor to perform the methods according to various embodiments of the present disclosure.

[0084] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are shown or described as block diagrams, flow charts, or by some other graphical representation, it is fully understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware, or a controller or other computing device, or some combination thereof, as non-limiting examples.

[0085] Computer-readable program instructions or computer program products for implementing various embodiments of the present disclosure may also be stored in the cloud, and when needed, users may access the computer-readable program instructions stored in the cloud and used to implement the embodiments of the present disclosure through mobile Internet, fixed network or other networks, thereby realizing the technical solutions disclosed in accordance with the various embodiments of the present disclosure.

[0086] Although the embodiments of the present disclosure have been described with reference to several specific embodiments, it should be fully understood that the embodiments of the present disclosure are not limited to the specific embodiments disclosed. The embodiments of the present disclosure are intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the claims is to be accorded the broadest interpretation and is therefore intended to encompass all such modifications and equivalent structures and functions.

Claims

1. A method for simulating an automated guided vehicle (AGV) system, the method comprising: Obtain sensor data of the virtual AGV in the simulation environment; Determining a device state of a virtual device interacting with the virtual AGV in the simulation environment; sending the sensor data and the device status to the AGV system, and receiving AGV motion control information and device operation information from the AGV system, wherein the AGV motion control information and the device operation information are generated by the AGV system based on the sensor data and the device status; and controlling the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information, respectively; Wherein, determining the device status of the virtual device interacting with the virtual AGV in the simulation environment further includes: obtaining state data of the virtual device based on a sensor model of the virtual device, the state data representing relative motion between moving parts of the virtual device, wherein the sensor model of the virtual device includes positions and parameters of virtual sensors of the virtual device; and The device status of the virtual device is determined based on the status data.

2. The method according to claim 1, wherein Obtaining sensor data of the virtual AGV in the simulation environment further includes: The sensor data is obtained based on a sensor model of the virtual AGV, the sensor model of the virtual AGV including positions and parameters of virtual sensors of the virtual AGV.

3. The method according to claim 1, wherein The controlling the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information further includes: determining a next position of a moving part of the virtual AGV based on the AGV motion control information, a kinematic model of the virtual AGV, and a current position of a moving part of the virtual AGV; and The next position of the moving part of the virtual device is determined based on the device operation information, the kinematic model of the virtual device and the current position of the moving part of the virtual device, wherein: The kinematic models of the virtual AGV and the virtual device respectively include motion relationships and motion parameters of moving parts of the virtual AGV and the virtual device.

4. The method according to claim 1, further comprising: Determine whether the virtual AGV collides with or is too close to a virtual device and / or environmental objects surrounding the virtual AGV.

5. The method according to claim 4, wherein Determining whether the virtual AGV collides with or is too close to a virtual device and / or an environmental object surrounding the virtual AGV further includes: determining a distance between the virtual AGV and the virtual device and / or the environmental objects surrounding the virtual AGV based on the 3D model of the virtual AGV and the 3D models of the virtual device and / or the environmental objects surrounding the virtual AGV; and Determining whether the virtual AGV collides with or is too close to the virtual device and / or the environmental objects around the virtual AGV based on the determined distance, wherein: The 3D model of the virtual AGV and the 3D models of the virtual device and / or the environmental objects around the virtual AGV respectively include contour data of the virtual AGV and the virtual device and / or the environmental objects around the virtual AGV.

6. The method according to claim 1, further comprising: According to the 3D models of all virtual AGVs, virtual devices and environmental objects in the simulation environment, 3D images of all the virtual AGVs, virtual devices and environmental objects are displayed through a display interface.

7. A device for simulating an automated guided vehicle (AGV) system, the device comprising: a sensor detection unit configured to obtain sensor data of a virtual AGV in a simulation environment; a state determination unit configured to determine a device state of a virtual device interacting with the virtual AGV in the simulation environment; a data communication unit configured to transmit the sensor data and the device status to the AGV system, and receive AGV motion control information and device operation information from the AGV system, wherein the AGV motion control information and the device operation information are generated by the AGV system based on the sensor data and the device status; and a motion control unit configured to control the motion of the virtual AGV and the virtual device in the simulation environment based on the AGV motion control information and the device operation information, respectively; Wherein, the state determination unit is further configured to: obtaining state data of the virtual device based on a sensor model of the virtual device, the state data representing relative motion between moving parts of the virtual device, wherein the sensor model of the virtual device includes positions and parameters of virtual sensors of the virtual device; and The device status of the virtual device is determined based on the status data.

8. The device according to claim 7, wherein The sensor detection unit is further configured to: The sensor data is obtained based on a sensor model of the virtual AGV, the sensor model of the virtual AGV including positions and parameters of virtual sensors of the virtual AGV.

9. The device according to claim 7, wherein The motion control unit is further configured to: determining a next position of a moving part of the virtual AGV based on the AGV motion control information, a kinematic model of the virtual AGV, and a current position of a moving part of the virtual AGV; and The next position of the moving part of the virtual device is determined based on the device operation information, the kinematic model of the virtual device and the current position of the moving part of the virtual device, wherein: The kinematic models of the virtual AGV and the virtual device respectively include motion relationships and motion parameters of moving parts of the virtual AGV and the virtual device. 10 . The apparatus according to claim 7 , further comprising a collision detection unit configured to determine whether the virtual AGV collides with or is too close to a virtual device and / or an environmental object surrounding the virtual AGV.

11. The device according to claim 10, wherein The collision detection unit is further configured to: determining a distance between the virtual AGV and the virtual device and / or the environmental objects surrounding the virtual AGV based on the 3D model of the virtual AGV and the 3D models of the virtual device and / or the environmental objects surrounding the virtual AGV; as well as Determining whether the virtual AGV collides with or is too close to the virtual device and / or the environmental objects around the virtual AGV based on the determined distance, wherein: The 3D model of the virtual AGV and the 3D models of the virtual device and / or the environmental objects around the virtual AGV respectively include contour data of the virtual AGV and the virtual device and / or the environmental objects around the virtual AGV.

12. The device according to claim 7 further includes a 3D rendering unit, which is configured to display 3D images of all virtual AGVs, virtual devices and environmental objects in the simulation environment through a display interface based on the 3D models of all virtual AGVs, virtual devices and environmental objects.

13. A computing device comprising: processor; as well as A memory configured to store computer-executable instructions which, when executed, cause the processor to perform the method according to any one of claims 1 to 6.

14. A computer-readable storage medium comprising computer-executable instructions stored therein, wherein the computer-executable instructions are used to execute the method according to any one of claims 1 to 6.

15. A computer program product tangibly stored in a computer-readable storage medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to any one of claims 1 to 6.

Images