Method for controlling an automated guided vehicle and control system suitable for performing the method

Abstract

This invention provides a method for controlling an automated guided vehicle (AGV) to transport at least two loads from a load pickup area, wherein the at least two loads will be placed in corresponding loading areas, to an operating area, and a control system suitable for performing the method. The method includes: obtaining information at least regarding pickup locations, the number and size of the loads to be transported; scanning the operating area to at least identify the spatial dimensions of the loading area; generating a loading pattern for placing the at least two loads within the corresponding loading areas within the virtual boundaries of the operating area; and executing the loading pattern until the loading task of transporting the at least two loads to the operating area is completed. Scanning the operating area includes: obtaining information regarding at least three corners of a defining polygon, and wherein optionally, the polygon of the operating area is added to the pickup area before generating the loading pattern.

Description

Technical Field

[0001] This invention generally relates to the loading and unloading of materials onto / from trucks or containers using autonomous forklifts or other autonomous vehicles when rear access is required to place materials inside rails or containers (loading) or to remove materials from said rails or containers (unloading). More specifically, it provides an apparatus and method for automated rear-end palletized cargo loading and unloading onto freight trucks or containers using an unmanned forklift loader. In addition to trucks and containers, the invention is also applicable to other types of freight transport vehicles, such as trailers or vans. Access to the freight transport vehicle is from the rear (as in the case of a truck) or at a loading area (such as a loading dock). During loading or unloading, materials or cargo are loaded into or removed from the freight transport vehicle. Furthermore, preferably, the cargo or materials are palletized, enabling the unmanned forklift to load and unload the freight transport vehicle. Background Technology

[0002] Currently, various automated and autonomous transport systems exist that can move goods from one location to another within a production or storage area, including transporting goods placed on pallets. Some of these automated transport systems include forks mounted at the front or rear of the vehicle (i.e., forklift). There are also known forklifts capable of loading or unloading goods from the side or rear entrance of a truck when it is parked at a loading gate. While the latter provides a solution, it is primarily designed for forklifts specifically designed to transport two loads simultaneously and equipped with tilting and lateral shifting mechanisms to shift the load to one side when traveling only parallel to the wall. These forklifts are balanced. Furthermore, proposed methods rely on motor current or pressure sensors to sense when a load impacts another load in the row or the end wall to place the load. To overcome sensing limitations, such solutions typically remove two loads picked up from the longer side of the pallet (only applicable to balanced vehicles without wheels on the forks) to completely fill the row in the trailer and do not handle situations where pallets need to be placed between or adjacent to each other in the same row. When handling heavy loads, such vehicles generate high pressure on the surface of the trailer, which can cause damage. Furthermore, these automated vehicles require specially prepared loads for pickup, necessitating infrastructure investment, which may not always be feasible due to space constraints or existing automated production lines.

[0003] In practice, forklifts capable of transporting only single loads and lacking tilting and, especially, shifting mechanisms are often used. Furthermore, loads frequently jam and require adjustment when they are not properly positioned and need to be placed close together in rows typically containing more than two loads. In such cases, relying on current or pressure sensors will render the loading task impossible. This also includes reverse operations, i.e., unloading similarly positioned goods from the rear of a truck. Simpler forklifts for loading or unloading operations are cheaper and can be used successfully and efficiently manually. Typically, loads are picked up from the shorter side of the pallet, where a slot exists that also allows unbalanced forklifts with wheels on their forks to enter and engage with the load. Therefore, to fully fill a trailer, such loads must be placed inside the trailer in rows of three. In larger transport vehicles such as railcars, such rows can be even larger, regardless of which side the pallet is picked up from. In this case, a common problem is that one pallet is placed very close to another, requiring not only high precision but also advanced sensing and load transfer technology. Loads may not be properly positioned, may jam, and then may require adjustment. Typically, the space between trays is limited to near 0mm.

[0004] In addition to the issues mentioned above, loading pallets also presents challenges. Although these operations are mostly performed manually, they require precision and a certain level of experience to place the pallets very close to each other. Often, the space between pallets is limited to near 0mm, and the loading on the pallets is not perfectly formed. This necessitates special techniques to push the pallets in, often requiring force to remove and reinsert them.

[0005] Furthermore, if the rows are not properly formed from the beginning, the forklift operator may only realize this late (when almost all the rows are formed and the last row is full). As a result, the doors cannot be closed. In this situation, the entire load must be removed and reloaded into the truck or container.

[0006] Currently, in most production or logistics facilities, loading goods onto or receiving them from and thus unloading them from transport vehicles (trucks or containers) at their final destination is still done manually or with the aid of complex loading / unloading mechanical systems installed inside such trucks or containers, for example, as shown in https: / / www.youtube.com / watch?v=pbpvyZgZgL0&list=PL2kviFOXIFHZAt1UHi aYuvZe5RzmABCJk&index=10&t=0s.

[0007] Document US 8,192,137 B2 discloses a system and method primarily designed for forklifts with two pairs of forks, which, in addition to standard vertical displacement (lifting), can also be horizontally displaced and tilted. Single-pair forklift vehicles require lateral displacement mechanisms to densely place loads. The method according to this document can only travel in a straight line parallel to the walls inside the transport device. Summary of the Invention

[0008] Technical issues

[0009] The object of this invention is to provide an apparatus and method with improved efficiency during loading and / or unloading of materials relative to a receiver. Furthermore, a highly efficient unmanned apparatus and method will be provided, wherein no modifications to trucks and / or containers are required, and preferably no special mechanical equipment is needed. This is also applicable to other freight transport devices, such as trailers and trucks.

[0010] Solution to the problem

[0011] This objective is achieved by the technical solution of this invention. Further aspects are defined in the specific clauses.

[0012] According to a first aspect of the invention, a method is provided for controlling an automated guided vehicle (AGV) or autonomous mobile robot (AMR) to transport at least two loads from a load pickup area, wherein the at least two loads will be placed in corresponding loading areas, to an operating area, wherein the method includes the steps of: picking up a first load in the load pickup area using the AGV; guiding the AGV carrying the first load from the load pickup area to the operating area by means of a guiding device; moving the AGV in the operating area to map a virtual boundary of the operating area, wherein the at least two loads will be placed in the corresponding loading areas within the virtual boundary; generating a loading pattern for placing the at least two loads in the corresponding loading areas within the virtual boundary of the operating area, and generating a loading pattern in which the AGV must be aligned with the first load. Each of at least two loads travels together to place the at least two loads in the corresponding loading area using a travel trajectory; the first load is placed in the corresponding loading area based on the generated loading pattern and the generated travel trajectory of the first load; the operating area is mapped with the first load placed in the corresponding loading area, and it is verified whether the first load in the corresponding loading area corresponds to the loading pattern in a manner that the at least one other load can be placed according to the loading pattern; and if the first load in the corresponding loading area does not correspond to the loading pattern in a manner that the at least one other load can be placed according to the loading pattern, the position and / or orientation of the first load is corrected in a manner that the at least one other load can be placed according to the loading pattern. Instead of the operating area, a zone of a freight transport device can be used.

[0013] In the first aspect of the method, the step of moving the vehicle in the operating area and generating a loading pattern can occur during the placement operation of the first load.

[0014] According to a second aspect relying on the first aspect, a method for controlling an AGV is provided, wherein the steps of guiding the AGV include: navigating through a position synchronization checkpoint and entering a confined space, in which the operating area is defined by a rear entrance or loading door. In this way, high-speed loading of the AGV is possible.

[0015] According to the third aspect, a control system suitable for performing the method of the first aspect is provided.

[0016] According to the fourth aspect, an automated guided vehicle with a control system according to the third aspect is provided.

[0017] According to the fifth aspect, a control kit is provided that is suitable for installation in a non-automatically guided vehicle to enable such vehicle to perform the methods of the first aspect in an automatic manner.

[0018] Using the technology of this invention, placement density can be automatically monitored. This improves the efficiency of loading tasks and saves time. Furthermore, the solution of this invention automates the final step of production.

[0019] The proposed solution requires no special equipment or any modification to the load transport container or operating environment due to natural navigation, and allows the use of driverless forklifts or other driverless transport vehicles with dimensions capable of rear-entry into cargo containers or trucks. In this configuration, trucks typically park at loading bays, allowing direct access to the inner area via ramps for loading; containers can be placed on the ground or loaded onto container trucks, and can also be parked at loading gates of logistics facilities or loading / receiving areas of production facilities.

[0020] In the case of loading tasks, automated forklifts equipped with the necessary sensing and computing hardware and running autonomous navigation and application software can sense the interior of the cargo container and identify its size, orientation, and offset (if it is not perfectly parked at the loading door or has shifted from its intended position). Based on the loading task received from the server, which includes information about the size and quantity of goods to be loaded from the pick-up area, the loading procedure is calculated, and the loading pattern and path, i.e., planning, are calculated and executed. During the execution of the loading task, goods are picked up from the defined location and loaded tightly together inside the truck container. The system can detect if the goods are not properly formed to fit tightly in the space. Detection is equally preferred during both pick-up and loading tasks. During pallet picking, the formation of the load is determined. If the load is poorly formed, the pallet may be rejected and the task suspended. Subsequently, the monitoring system / server is notified.

[0021] During and after each placement, the placement quality is automatically controlled, and corrections can be attempted if the results are unsatisfactory. To better understand "during each placement," this refers to a situation where the load is detected as stuck, meaning the forklift cannot push the load to the required position and needs to reverse and attempt correction.

[0022] For unloading tasks, the system automatically identifies the size, orientation, and offset of docked or placed containers at the unloading point (or gate), and automatically identifies and maps the loaded cargo pattern and the virtual boundaries of the operating area with the help of the onboard sensing and computing equipment of the automated forklift vehicle (AGV). Therefore, unloading planning is calculated and executed in a manner that involves picking up cargo from the cargo transport containers / trucks via the rear / unloading entrance, including entering the container / truck area to pick up the cargo and moving it to a sequential location received from the server, where the sequence is defined in the unloading task. The fleet management software participates as follows: it only sends information about the load, such as the minimum number, form, and size (if any) of pallets, gate number or container location information, and the unloading location or initial coordinates and the required storage pattern. Everything else is calculated on the AGV, allowing the fleet management software to only play a supervisory role.

[0023] The main advantage of automating loading and unloading tasks using the unmanned automated vehicles according to the invention is that truck driver rest time is optimized, allowing truck drivers the necessary rest time before arriving at their next trip while waiting for goods to be loaded in parallel. Furthermore, transport weighting time and loading door usage can be optimized through a predictable deterministic process. This is an additional step required in automating the delivery process for the automation of the entire transport or logistics chain, including unmanned trucks. This includes the emergence of unmanned trailers or other unmanned freight transport devices. This step in delivery chain automation needs to be implemented correctly to serve as many scenarios as possible.

[0024] According to a first additional aspect of the invention, a system for automating material / cargo loading and unloading can be provided, comprising a self-contained, automatically driven robotic material handling vehicle, including:

[0025] a) It features both automatic and manual control via wired operation;

[0026] b) Location determination subsystem;

[0027] c) Approach obstacle detection and avoidance subsystem;

[0028] d) Loading / unloading sequence, pattern, and trajectory planner;

[0029] e) Trajectory following execution controller;

[0030] f) Load the location identification and monitoring subsystem;

[0031] g) Loading / unloading task request client;

[0032] And h) Loading and unloading task management subsystem.

[0033] In a system based on a second additional aspect that depends on a first additional aspect, the mapping may include increased planning execution precision by dynamically changing regions from task to task, where the plan is a list of sub-plans consisting of waypoints for each load of the task.

[0034] In a system based on a third additional aspect that depends on one of the other additional aspects, the map may include more than one planning execution area for improved accuracy, including location synchronization markers for positioning error elimination.

[0035] In a system based on a fourth additional aspect that depends on one of the other additional aspects, the vehicle's database includes vehicle mission planning, tracking data, and vehicle status.

[0036] According to a fifth additional aspect of a system that depends on one of the other additional aspects, it may include sensors and processing electronics (referred to as a sensor suite) that enable material transport vehicles to navigate in confined environments, such as inside a truck or container, including entering through a rear entrance or loading door and navigating through location-synchronized checkpoints.

[0037] According to a sixth additional aspect of a system that depends on one of the other additional aspects, it may include sensors and processing electronics (referred to as a sensor kit), which may be installed in existing commercial material handling vehicles equipped with forks or other means of transporting goods or materials.

[0038] According to the seventh additional aspect of the system which depends on one of the other additional aspects, it may include sensors and processing electronics (referred to as a sensor suite) wherein the width of the material transport vehicle is required to be less than or equal to the width of the material being transported.

[0039] According to the eighth additional aspect of the system, which depends on one of the other additional aspects, it may include an autonomously driven robotic material handling vehicle with sensors and processing electronics, which enables the vehicle to navigate in confined environments such as inside a truck or container, including entering through a rear entrance or loading door and navigating through location-synchronized checkpoints.

[0040] According to a ninth additional aspect of a system that depends on one of the other additional aspects, it may include an autonomously driven robotic material handling vehicle having sensors and processing electronics in the form of an autonomous forklift and pallet truck.

[0041] According to the tenth additional aspect of a system that depends on one of the other additional aspects, sensors and processing electronics (referred to as sensor suites) can be included in a modular, mission-specific setup within the architecture, which can be determined in a general location.

[0042] According to the eleventh additional aspect, which depends on one of the other additional aspects, a system may include an autonomously driven robotic material handling vehicle having modular, mission-specific sensors and processing electronics within a general location-determining architecture.

[0043] In a system based on a twelfth additional aspect that depends on one of the other additional aspects, one or more of the following sensors are incorporated into the general position determination architecture:

[0044] IMU;

[0045] Magnetometer;

[0046] Differential odometer (via magnetic or optical encoder)

[0047] Visual benchmark;

[0048] 2D rangefinder (2D LIDAR);

[0049] 3D rangefinder (3D LIDAR);

[0050] Single-range sensor;

[0051] Optical sensors (single camera or stereo pair)

[0052] The optical sensors (single camera or stereo pair) are passive, or have a built-in light-emitting system for depth / range calculation.

[0053] According to the twelfth additional aspect, the system is not limited to differential odometers; other odometers may also be used. Visual references and optical sensors may be optional.

[0054] According to the thirteenth additional aspect of a system that depends on one of the other additional aspects, it may include sensors and processing electronics (referred to as a sensor suite) for reading and interpreting visual codes that encode location and other relevant data as a reference to determine indoor location.

[0055] According to the fourteenth additional aspect, which depends on one of the other additional aspects, a system may include an autonomously driven robotic material handling vehicle having sensors and processing electronics to read and interpret visual codes that encode position and other relevant data as a basis to determine indoor location.

[0056] According to the method of the fifteenth additional aspect, for automatically controlling a vehicle to transport at least two loads from a load pickup area in which at least two loads will be placed in corresponding loading areas to an operating area, said method comprising the steps of: acquiring at least information about the pickup location, the quantity and size of the loads to be transported, and optionally about the loading areas; scanning the operating area to at least identify the spatial dimensions of the loading areas; generating a loading pattern for transporting at least two loads from the load pickup area to the loading area; said loading pattern including the target locations and target orientations of the vehicles to be sequentially arrived at; and executing the loading pattern until the loading task of transporting at least two loads to the operating area is completed. In this way, the transport of at least two loads can be performed efficiently while requiring a minimal amount of control structure.

[0057] In the method according to the sixteenth additional aspect, which relies on the fifteenth additional aspect, upon completion of the loading task, the vehicle reports successful execution of the loading task and navigates the vehicle to a predefined waiting position. Therefore, the loading process can be terminated in a very short time, while possessing appropriate information for further transport of available loads.

[0058] In the method according to the seventeenth additional aspect, which depends on the fifteenth or sixteenth additional aspect, if a failure occurs during the execution of the loading mode, recovery actions to correct the failure are performed, and if the correction of the failure fails, the failure is reported to the server or fleet management system. Using this method, appropriate measures can be taken immediately to ensure proper loading.

[0059] In the method according to the eighteenth additional aspect, which depends on the fifteenth to seventeenth additional aspects, scanning the loading area includes information about the offset dx, the distance dy between the load pickup area and the operating area, and optionally information about the angle α of the orientation difference between the load pickup area and the operating area. Therefore, efficient generation of the loading pattern is possible with minimal received information.

[0060] In an alternative to the eighteenth additional aspect, which relies on the fifteenth to seventeenth additional aspects, a method is provided in which scanning the operating area comprises: obtaining information about at least three corners of a defining polygon in which at least two loads will be placed, and wherein optionally, the polygon of the operating area is added to the picking area before generating the loading pattern for transporting the at least two loads from the load picking area to the loading area. In this way, the perimeter of the transportation system can be determined efficiently.

[0061] In this alternative of the eighteenth additional aspect, obtaining information about at least three corners of the defined polygon may include: using filtering techniques to determine a loading area that can be traversed with predefined precision, and optionally using ranging or image data processing to determine at least one wall of the polygon relative to the operating area, on which at least one of the at least two loads will be placed. Using this information, a safely traversable area can be appropriately defined, and verification of information about the transportation system can be performed in an efficient manner.

[0062] Furthermore, relative to the eighteenth additional aspect, before generating the loading pattern, the inclination between the operation area and the pickup area, and optionally the two-dimensional offset between the operation area and the pickup area, can be determined using the two corners of the polygon. This allows the operation area to be calculated with minimal computational effort.

[0063] In the method according to the nineteenth additional aspect, which relies on the fifteenth to eighteenth additional aspects, the loading pattern comprises sub-plans in the form of trajectories, where each sub-plan ends with a descent action with respect to the load. This segmentation enables efficient use of previously acquired information.

[0064] According to the twentieth additional aspect, a vehicle is capable of automatically transporting at least two loads from a load pickup area, wherein the at least two loads will be placed in corresponding loading areas, to an operating area. The vehicle includes: means for acquiring information at least regarding the pickup location, the quantity and size of the loads to be transported, and the loading area; means for scanning the loading area to at least identify the spatial dimensions of the loading area; means for generating a loading pattern for transporting the at least two loads from the load pickup area to the loading area, wherein the loading pattern includes target locations and target orientations of the vehicles to be sequentially reached; and means for executing the loading pattern to automatically transport the at least two loads to the operating area until the loading task is completed. Such a vehicle is capable of efficiently performing the method according to the fifteenth additional aspect.

[0065] According to the twentieth additional aspect, a vehicle is capable of automatically transporting at least two loads from a load pickup area to an operating area, wherein the at least two loads will be placed in corresponding loading areas. The vehicle includes: means for picking up a first load from the load pickup area using the vehicle; means for guiding the vehicle carrying the first load from the load pickup area to the operating area via a guiding device; means for moving the vehicle in the operating area to map a virtual boundary in the operating area during the placement operation of the first load, the at least two loads being placed in the corresponding loading area within the virtual boundary; means for generating a loading pattern during the placement operation of the first load, the loading pattern being used to place the at least two loads in the corresponding loading area within the vehicle boundary in the operating area, and generating a travel trajectory that the vehicle must travel with each of the at least two loads to place the at least two loads in the corresponding loading area; and generating a travel trajectory during the placement operation of the first load for... The vehicle is configured to: place the at least two loads in the corresponding loading area within the vehicle boundary of the operating area using a loading pattern; generate a travel trajectory in which the vehicle must travel with each of the at least two loads to place the at least two loads in the corresponding loading area; place the first load in the corresponding loading area based on the generated loading pattern and the generated travel trajectory of the first load; map the operating area with the first load placed in the corresponding loading area and verify whether the first load in the corresponding loading area corresponds to the loading pattern in a manner in which the at least one additional load can be placed according to the loading pattern; and correct the position and / or orientation of the first load in a manner in which the at least one additional load can be placed according to the loading pattern if the first load in the corresponding loading area does not correspond to the loading pattern in a manner in which the at least one additional load can be placed according to the loading pattern. Such a vehicle can efficiently perform the method according to the first aspect.

[0066] In the vehicle pursuant to the twenty-first additional aspect, which relies on the twenty additional aspect, the means for performing the loading mode is capable of identifying problems of incorrect load placement or insertion by using at least one 3D ranging or optical camera. Therefore, loading problems can be identified at an earlier point in time, and appropriate corrections can be triggered before other loading problems occur.

[0067] In the vehicle according to the twenty-second additional aspect, which relies on the twenty-first additional aspect, the device for executing the loading mode is capable of attempting to correct the load placement in case of incorrect load placement and problems with load insertion, and is configured to transmit the correction failure to a server or a monitoring fleet management system if the correction fails. This correction attempt avoids the transmission of unnecessary information and enables the vehicle to operate autonomously for as long as possible, which is useful.

[0068] In vehicles according to the twenty-third additional aspect which relies on the twenty-first or twenty-second additional aspect, the at least one 3D ranging or optical camera is actuable or retractable to change the viewpoint of the carried load and / or adjacent loads when performing the loading task. This can even be observed from the side of the vehicle to determine possible points of impact and to prevent collisions in the event that the load is not properly shaped, tilted, or displaced relative to the pallet on which it is placed.

[0069] The reverse operation, i.e., unloading, can be performed using the same sensors and methods specified above. However, when transporting the first load, the AGV / AMR does not scan the transport system; it needs to be empty, but scanned in the same way. The loading pattern can be known, for example, received from the server before the unloading operation begins, or it needs to be identified or inferred, in which case it is also generated. All other steps are the same. The surrounding corners are identified, and when the pattern comes from the server or is inferred, as already described, the pattern rotates around the corners to adapt to the perimeter of the transport vehicle, and now an unloading plan containing the exact same information (the set of trajectories and the final destination and orientation) is generated. Now it's reversed—the load moves from the transport system, not into it.

[0070] These conditions are reflected in the following vehicles according to the twenty-fourth additional aspect, which are capable of automatically transporting (preferably for unloading) at least two loads from an operating area where the at least two loads have been placed in the corresponding loading area to a load destination area, wherein the vehicle includes: means for acquiring at least information about the load destination area, about the quantity and size of the loads to be transported, and about the loading area; means for scanning the loading area or acquiring information about the loading area to at least identify the spatial dimensions of the loading area; means for generating a loading pattern for transporting the at least two loads from the loading area to the destination area, wherein the loading pattern includes the target locations and target orientations of the vehicles to be arrived at sequentially; and means for executing the loading pattern to automatically transport the at least two loads to the destination area until the loading task is completed.

[0071] According to the twenty-fifth additional aspect, a method for automatically controlling a vehicle to transport at least two loads from an operating area where the at least two loads have been placed in corresponding loading areas to a load destination area, wherein the method includes the steps of: acquiring information at least about the load destination area, about the quantity and size of the loads to be transported, and about the loading area; scanning the operating area or acquiring information about the operating area to at least identify the spatial dimensions of the loading area; generating a loading pattern for transporting the at least two loads from the loading area to the destination area, wherein the loading pattern includes target locations and target orientations of the vehicles to be sequentially reached; and executing the loading pattern to automatically transport the at least two loads to the destination area until the loading task is completed. Attached Figure Description

[0072] The foregoing objects, as well as other objects, features, and advantages of the present invention, will be described in more detail with reference to the accompanying drawings and corresponding detailed descriptions.

[0073] Figure 1A An automated guided vehicle (AGV) with a sensor kit according to the present invention is shown in a perspective view. Figure 1B The computing unit of the AGV to which the sensor is connected is shown.

[0074] Figure 2 The rangefinder field of view and measurement coverage in the lower part of the vehicle according to Figure 1 is shown in the second embodiment.

[0075] Figure 3 The possible scenarios for picking up and dropping off areas are shown.

[0076] Figure 4A and Figure 4B An operating area with virtual boundaries is shown, within which the loading area is defined.

[0077] Figure 5A and Figure 5B A method for controlling an AGV according to the present invention is shown.

[0078] Figure 6A and Figure 6B An automated guided vehicle (AGV) with a sensor kit according to the invention is shown in a perspective view and a side view according to a third embodiment.

[0079] Figure 7 An automated guided vehicle (AGV) with a sensor suite according to a fourth embodiment of the present invention is shown in a perspective view.

[0080] Figure 8 The possible scenarios for picking up and dropping off areas are shown. Detailed Implementation

[0081] This invention relates to the automated loading of materials / cargo inside trucks or containers upon entry into a transport area. Other freight transport devices, such as trailers or railcars, can be used instead of trucks or containers. Typically, lifting vehicles (usually forklifts, but also different types of automated transport platforms) for materials or cargo enter from the rear (e.g., trucks parked at the gate) for dense loading of goods. In another aspect of the invention, trailers or railcars can be parked at the gate or on a platform. The lifting vehicle is preferably the same width as or narrower than the width of the loaded goods / materials. The invention also includes the reverse operation under the same conditions, i.e., unloading. The method allows the use of existing transport platforms through sensor suite integration (modification) and automated lifting platforms specifically designed for such applications. Material transfer platforms can be used instead of transport platforms and lifting platforms.

[0082] The system includes a vehicle equipped with forks or other mechanisms for lifting and transporting goods, typically placed on pallets or otherwise securely fastened (preferably in a rectangular prism shape). The vehicle is equipped with sensing and computing devices (sensor suites) including: distance and optical sensors located in different zones of the vehicle to sense the surrounding environment for positioning the vehicle within the operating area and controlling load placement; and a computing unit for calculating positioning and navigation algorithms and communicating with a server / fleet management system. Preferably, the vehicle is a standalone unit and does not require external supervision to control its operation or the installation of any external tracking sensors. The vehicle receives task commands only from the supervision system or server and reports back task completion or failure (if applicable). In case of failure, the vehicle can be manually controlled, including remote manual control.

[0083] The first embodiment of the invention will now be described with reference to FIG1. ​​For safe navigation and sensing of the environment surrounding vehicle 1, ranging sensors 2a, 2b, and 2c (preferably distance sensors) are mounted in a manner that provides complete coverage in both directions of travel of vehicle 1. Multiple sensors with overlap are used to cover situations where one sensor in each direction cannot deliver the required field of view due to obstruction. The sensing technology does not affect the underlying control logic as long as it can provide accuracy similar to or better than LIDAR. Alternatively, sensors can be used to ensure the accuracy required for safe navigation and the desired application. This means that a camera can also be used as a ranging sensor when the distance measurement or attitude estimation derived through image processing meets the above requirements.

[0084] Figure 1BA computing unit for an AGV according to a first embodiment is shown, with sensors connected to the computing unit. The computing unit and sensors are part of an automation kit according to the present invention, which enables the following functions:

[0085] For the minimum functional setup, the following sensors are required:

[0086] 1. Front and rear LiDAR.

[0087] The front LIDAR is a LIDAR located on opposite sides of a pair of forks and is primarily used for rapid travel when unloaded and when carrying a load over long distances.

[0088] The rear lidar is located on the side above the load, above a pair of forks. It is primarily used for loading operations involving the load into trucks and containers.

[0089] The two LIDARs complement each other to improve the vehicle's attitude estimation in the operating area.

[0090] Both types of LIDAR can be 2D or 3D, or can be replaced by other sensors that can deliver distance data with similar quality to 2D or 3D LIDAR.

[0091] In another implementation, safety regulations require that one or a pair of rear-facing LiDARs be installed below the forks to ensure unobstructed visibility when transporting the load. Alternatively, the LiDARs may be mounted at an angle (2b) to the ground above the load. The LiDARs are not essential for the transport loading function, but are necessary for moving forward with the load / forks when other AGVs or humans are operating in the area.

[0092] 2. A rangefinder camera (also known as an RGB-D camera) is a device that delivers optical image data and dense distance measurements associated with image pixels. The camera is primarily used to monitor load placement, i.e., measuring the clearance between a load on the forks and an adjacent load in the transport vehicle. Additionally, the camera can be used to identify the shape of the load and measure the remaining clearance between loads after placement.

[0093] The camera is oriented to ensure a clear view of the end of the load on the forks and the orientation of adjacent objects. Depending on the type of automated vehicle used, the camera can be placed in a fixed position or actuated, for example, extending to one side of the vehicle when there is space on the side, and moving to a safe position when the vehicle is operating close to the walls of the transport vehicle (7a, 7b). Its field of view is large enough to see the area in front of the vehicle in the fork direction to identify the vehicle's geometry, such as the edges of the floor and walls and their connection points. Even in the absence of walls, the edges of the platform can be identified to aid in planning and attitude calculations.

[0094] In another embodiment, the ranging camera can be replaced by a pair of optical cameras (also known as a stereo pair), which provides the possibility of deriving distance data known in the art through algorithms. Alternatively, the pair of cameras can be a single camera with a distance sensor that measures the distance from the camera to the load to identify the scale and derive distance data also known in the art. A pair of cameras can also be provided, for example, on each side of an automated vehicle, with or without a distance sensor.

[0095] 3. IMU - Inertial Measurement Unit 303 (see...) Figure 6B It combines gyroscopes and accelerometers with a magnetic sensor, typically with three degrees of freedom. The IMU is used in sensor fusion algorithms to improve vehicle attitude estimation in the environment.

[0096] 4. When automating commercial vehicles, wheel encoders and steering encoders are typically part of the system. If they are not present, they need to be installed. Wheel encoders typically provide incremental scales that increase or decrease according to the direction of travel. Given encoder resolution and wheel diameter, the scale can be converted into speed. Steering encoders also provide absolute incremental scales or other signals that can be converted into the absolute steering angle of the steering wheel. Absolute means that even if the system is off, the data can always be converted into the steering angle.

[0097] 5. The current sensor is used to detect overload of the system when a load on the forks comes into contact with another load or the wall of the transport vehicle. Importantly, it also allows for the detection of a stuck load when attempting to position it properly. The current sensor is typically part of the vehicle and is usually present in the system directly or indirectly through RMS measurements of the drive motor phase current by the drive controller.

[0098] In another implementation, when the drive motor current cannot be measured due to technical reasons, a pressure sensor between the load and the vehicle can be installed on the forks to measure the pressure increase when the load comes into contact with an obstacle or gets stuck, for the same purpose.

[0099] 6. To propel the vehicle, the guidance system calculates the required speed and communicates it as a signal understandable to the speed controller. Therefore, the vehicle's linear speed is controlled.

[0100] In another implementation, when the differential drive system is used with two independently driven wheels, the desired speed is calculated and transmitted to the corresponding speed controller, thereby controlling both the linear speed and angular speed of the vehicle.

[0101] 7. In order to steer the vehicle using the steering wheel, the steering system calculates the required steering angle and communicates it in a signal that the steering controller can understand, thereby controlling the vehicle's angular velocity.

[0102] 8. In order to pick up and place loads, the guidance system calculates lifting commands and communicates them in signals that the lifting controller can understand.

[0103] If it is necessary to measure the height of the forks for a specific loading operation, a linear encoder or any other type of sensor not shown in the figure can be installed.

[0104] Figure 2 (A second embodiment of the invention) illustrates the field-of-view coverage of distance sensors 12c, 12e, and 12d when fork-side sensors 12d and 12e, arranged on opposite sides of vehicle 10, are obstructed in the direction of observation. To cover situations where one sensor in each direction cannot deliver the required field of view due to obstruction, multiple overlapping sensors are used. Figure 2 Sensors 12d and 12e, which point in the same direction and can be optional sensors for lateral distance, are used as examples. Here, it is important for safe navigation that sensors 12d and 12e are mounted in a lower position at the wheel level of vehicle 10, while sensors mounted higher provide for more specific needs. A similar approach can be used for other sensor positions to achieve the same goal. The minimum required vertical field of view is planar 2D, but this can be extended to higher vertical angles when using 3D ranging sensors or equivalents (e.g., RGBD cameras).

[0105] Figure 2 The rangefinder's field of view 14a, 14b, 14c, 14d, 14e and the measurement coverage 16a, 16b, 16c, 16d, 16e in the lower vehicle area of ​​Figure 1 are shown.

[0106] In the first embodiment, the ranging sensor 2a, located on the upper side of the vehicle, is positioned to ensure forward visibility when a load is placed on the vehicle 1. In the second embodiment, a ranging sensor similar to the ranging sensor 2a in the first embodiment can also be installed.

[0107] If the ranging sensor 2a is 2D or does not have sufficient vertical field of view, it can provide a similar... Figure 6A and Figure 6BThe tilting mechanism in the third embodiment enables a full vertical scan of the load placement and, when functional safety standards need to be met, also allows for vertical obstacle / object detection while navigating forward with the load. Instead of or in addition to the tilting mechanism, optical or 3D ranging sensors with a narrower field of view, such as RGBD or ToF (Time-of-Flight) cameras (2f), can be used. This capability helps ensure the load is correctly placed in the truck or container for loading operations and identifies load placement and, if necessary, forklift slots and load formation during pick-up or unloading tasks. The sensor's position above the load depends on the length and height of the load being delivered for the loading or unloading application and can be adjusted manually or automatically.

[0108] The upper ranging sensor 2a of the first embodiment, with an optional tilting mechanism, has no horizontal view obstruction in at least one horizontal plane. This means that the horizontal field of view is defined by the sensor characteristics and can typically reach 360 degrees. Thus, if the field of view is wide enough, it can provide a perfect auxiliary (secondary) positioning sensor. According to functional safety requirements, the upper sensor 2a can be used for independent positioning calculations, which are cross-checked with the primary calculation at fixed time intervals. If the mismatch exceeds a threshold, an operational error is issued.

[0109] When entering a container or truck along with a vehicle according to the first embodiment, the upper ranging sensor 2a is the primary sensor for precise positioning within the truck / container area. Placing the sensor 2a horizontally above the load height of the container ensures an unobstructed view of the perimeter of the container's interior space. To ensure the ability to transport loads of varying heights, the sensor 2a is preferably placed at the maximum permitted height for entry into the truck or container, meaning the height is typically no higher than 2.2m.

[0110] To ensure proper load / pallet placement on the forks or cargo box, an additional optical sensor 2f or a pair of sensors can be installed on the top. This sensor or sensor pair is installed in a manner that ensures a complete view of the load from the top while avoiding the aforementioned maximum permissible height increase.

[0111] If the fork mechanism is used together with the vehicle 1 of the first embodiment to pick up pallets or loads, two ranging / distance sensors 3a and 3b are preferably positioned on the underside of the vehicle carrying the load (e.g., at the end of the forks of a lifting vehicle equipped with forks). Sensors 3a and 3b may be ranging-only, or may incorporate a combination of optical and ranging / distance measurement technologies. These ranging sensors 3a and 3b may optionally include a camera. In addition to measuring distances associated with approaching objects, optical sensors add the possibility of better identifying contours and openings (e.g., fork slots).

[0112] In cases where the upper ranging sensors 2a and 2f of the first embodiment cannot provide sufficient accuracy to measure distances to the sides of the container or truck, auxiliary ranging / distance measurement sensors 5a and 5b can be mounted on both sides of the lifting vehicle 1 along its width. Sensors 5a and 5b are mounted in a manner that ensures effective distance measurements are delivered even when the sides of the lifting vehicle are in contact with the wall of the container / truck, while traveling along one side of the container / truck.

[0113] To support global attitude estimation of the vehicle (i.e., in coordinates of the operational space, where attitude is related to position and orientation), additional sensors (such as wheel encoders) 6a, 6b, and IMU 7 are used. The information from these sensors 6a, 6b, and 7, when combined, delivers a locally consistent attitude estimate, preferably in the robot's coordinate system, which can be converted to the desired global coordinate system. Sensors 6a and 6b are preferably wheel encoders integrated with the wheels. Alternatively, separate wheels with encoders can be provided attached to the vehicle body. If the vehicle has combined steering and propulsion wheels and there are no stringent safety requirements necessitating redundant wheel encoders, sensors 6a and 6b can be omitted, and steering encoders and wheel encoders (such as...) can be used. Figure 6B Those 306, 307 shown in the alternative implementation schemes.

[0114] This attitude estimate drifts over time but is corrected by a global attitude estimate. Sensors 6a and 6b can be optical or magnetic encoders, preferably on both sides of vehicle 1, and sensor 7 is an IMU (Inertial Measurement Unit), which can be integrated into the computing module or located at other convenient locations in vehicle 1 in the first or second embodiment.

[0115] The vehicle maintains its position and orientation, i.e., locates itself in the operating environment, by means of onboard ranging sensors 2a, 2b, 2c or equivalent sensors, optical or magnetic encoders 6a and 6b, and IMU 7. The IMU can be placed in any part of the vehicle, including the computing module.

[0116] The internal representation of the environment (map) is loaded from the server or acquired during the operation preparation process. The operation area is actually subdivided into several sectors, each with different accuracy requirements for maintaining positioning and location. Figure 3 The largest area shown is marked (A), and it is the general operating area for automated vehicles with general operating accuracy requirements.

[0117] Figure 3Zones (B) and (C) are zones with increased precision requirements. These zones (B) and (C) are the pick (unload) zone and the drop (load) zone, respectively. There is no fundamental difference between zones (B) and (C), as they can be interchanged depending on the task, such as whether a load operation or an unload operation is performed.

[0118] A preferred principle is that the load needs to be picked up precisely from one place and placed precisely in another. Figure 3 Zone (B) in the map represents the container or truck zone, where the angle (α) indicates that the container at the loading door is not exactly perpendicular to the wall / door. When creating the map, the doors are typically closed, so knowing the angle (α) in addition to the container or truck dimensions can be helpful. However, the first implementation is not limited to knowing the angle (α).

[0119] When loading or unloading operations are performed using long-haul transport vehicles parked at the gate, the gate coordinates are known a priori. Therefore, estimating the angle (α) (if present), lateral offset, and the width and length of the container / truck is sufficient. In fact, the list of container / truck lengths and widths is usually known a priori, so only a match needs to be found. When the container is not placed at the gate, X and Y coordinates need to be provided or established separately, but in most cases, the X and Y coordinates are known a priori, or the offset can be automatically identified if needed. At any time, if a match is not found, the container / truck area can be mapped and a virtual boundary 110 can be applied to strictly define the operational boundaries, even in the absence of one or more walls.

[0120] To ensure that the required accuracy for positioning and navigation can be achieved in zones (B) or (C), special attitude synchronization markers 30a and 30b for zone (B) and 32a and 32b for zone (C) can be installed on the floor or wall near the entrance points of these zones. These markers 30a, 30b and 32a, 32b are preferably visual markers that can be square reference markers, for example, as shown in S. Garrido-Jurado, R. Munoz-Salinas, FJ Madrid-Cuevas and MJ Marin-Jimenez. 2014. "Automatic generation and detection of highly reliable fiducial markers underocclusion". Pattern Recogn. 47, 6 (June 2014), 2280-2292. DOI = 10.1016 / j.patcog.2014.01.005, and in Arllco: a minimal library for Augmented Reality applications based on OpenCV, http: / / www.uco.es / investiga / grupos / ava / node / 26. Alternatively, other markers that allow for relative attitude estimation (coordinates X, Y and orientation) of the vehicle via image processing algorithms can be used.

[0121] The positions of the markers in the global operating coordinate system are known to the vehicle system. To detect markers 30a, 30b and / or 32a, 32b, if the markers are mounted on the floor near zones (B) and / or (C), camera 8 can be used. Alternatively, the camera can be mounted at a location on vehicle 1 where easy detection of markers 30a, 30b and / or 32a, 32b is possible. If the camera is used in conjunction with range sensors 3a, 3b, markers on the wall can also be detected without installing an additional camera.

[0122] The loading and unloading operations are then described in more detail.

[0123] The loading operation begins with a task order retrieved from a server or fleet management system. Other information includes the pick-up location, the quantity and size of the cargo / material to be loaded, and the loading gate number or container location. Before starting the loading operation, containers or trucks at the gate are automatically scanned to identify space dimensions and optional angles (α) and offsets. Based on the identified container / truck dimensions and cargo / material dimensions and quantities, a loading pattern, or plan, is generated. The overall plan is a list of sub-plans, preferably in the form of trajectories, used to move each individual load from the pick-up location to the appropriate position in the truck or container based on the generated loading pattern. Each sub-plan is a set of points describing the vehicle's target location and orientation, i.e., a set of postures to be reached sequentially. During the execution of each sub-plan, a pick-up action is defined. Each sub-plan ends with a drop-off action. The execution of the overall plan is managed by a task management algorithm, which reports any failures to the server or fleet management system and, if possible, performs recovery actions. Upon completion of the loading task, the vehicle reports successful execution and navigates to the defined waiting position.

[0124] In navigation within confined spaces (such as inside a container or truck) where increased accuracy is required (B), the top-positioned ranging sensor 2a of the first embodiment is used to calculate the precise relative position based on the known geometry of the container / railway. If a 3D ranging camera is installed in place of the tilting mechanism described above, or in addition to the tilting mechanism described above, this 3D ranging camera can be used as an auxiliary sensor to assist in making this precise local estimate. The local estimate is then converted to a global coordinate system to perform correct loading planning.

[0125] Each load placement inside the container or truck is verified after each drop / placement operation using an upper ranging sensor 2a of the first embodiment, which has a tilting mechanism or a 3D ranging camera mounted in addition to a tilting mechanism. If the load placement is incorrect, an attempt is made to correct it. If correction fails, the loading task is paused, and the relevant fault is communicated to the server or the monitoring fleet management system. Manual correction can be attempted, after which the loading schedule can be resumed using the next load in the list.

[0126] During planning execution, the area along the travel direction is monitored relative to the presence of obstacles. If an obstacle appears in a potential collision zone, planning execution is paused. If the obstacle does not disappear within a defined time period and remains stationary, a replanning attempt is made. If replanning fails or the new plan (e.g., trajectory) cannot be accurately followed, the loading operation is stopped, and a failure is reported.

[0127] The unloading operation begins with a task order, received from a server or fleet management system, similar to the loading operation. This task order contains information about the unloading gate number or container location, and information about the load (including quantity, size, and placement location). The truck or container is scanned to (if applicable) find the angle (α) and offset and verify the internal space dimensions. The load is then scanned using an upper ranging sensor 2a and a tilting mechanism, or alternatively, a 3D ranging camera 2f mounted in addition to the tilting mechanism. A placement identification algorithm identifies the load placement based on the ranging and / or additional image data, and compares the load placement with information received from the server. The vehicle then calculates an unloading plan, which, for example, consists of sub-plans that constitute the transport trajectory for each individual load, as described in the loading operation above.

[0128] exist Figure 3 In the diagram, areas B and C are shown as interchangeable pick (unload) and drop (load) areas.

[0129] exist Figure 4A and Figure 4B The image shows a loading area 100 with a virtual boundary 110. The virtual boundary 110 indicates the boundary where a load can be placed. In this example, a first row with three columns is shown. This means that loading areas 120, 122, and 124 are defined side by side and in such a way that the automated guided vehicle (AGV) according to the invention can place a suitable load. Loading areas 120, 122, and 124 are defined near the front portion 112 of the virtual boundary 110, and preferably in such a way that they extend over the entire front portion 112.

[0130] In order to transport the load to the corresponding loading areas 120, 122 and 124, tracks 130, 132 and 134 are... Figure 4B As shown in the diagram. These tracks enable the AGV according to the invention to place loads on loading areas 120, 122 and 124.

[0131] Figure 5 illustrates an example of a method for controlling an AGV to transport loads from a load pickup area to an operating area on which at least two loads will be placed. The method is performed in the following manner:

[0132] After the method begins in step S10, the AGV navigates to the load pickup area (step S20). While in the load pickup area, the AGV verifies whether the task to be picked up and transported to the operation area contains information about the load size (step S30). If the task does not contain information about the load size, the load size in the load pickup area is identified in step S40, and the method continues to step S50. If the task contains information about the load size, the method immediately proceeds to step S50. In this step S50, the AGV picks up the load. Subsequently, the load can be transported by the AGV.

[0133] In the next step S60, it is verified whether the map of the operation area has been expanded to include the transportation space, and it is also verified whether a loading plan exists. If it is determined in step S60 that the map of the operation area has been expanded to include the transportation space and that a loading plan exists, the method proceeds to step S150. If the map of the operation area has not yet been expanded to include the transportation space or that a loading plan does not yet exist, the method proceeds to step S70, where the AGV receives the task and navigates within the unknown space to map the area. In other words, the AGV navigates within the operation area on which the load is to be placed to obtain a map of the area.

[0134] In the subsequent step S80, the map of the operating area is expanded using the obtained observations, and in step S90, virtual boundaries defining the loading area are identified and assigned. In the subsequent step S100, the geometry of the area is identified to help track the position or orientation of the AGV, and in the subsequent step S110, a loading pattern and travel trajectory are generated that enable the AGV to transport the load from the pick-up area to the loading area.

[0135] In the subsequent step S120, the validity of the loading plan is verified, because for an invalid loading plan, it is impossible to load the loading area in an effective manner.

[0136] If the verification in step S120 indicates that the loading plan is invalid, a backup of the safety distance is generated or moved back until an optional synchronization marker for the loading area is visible, for example... Figure 3 Synchronization markers 30a and 30b. After step S130, jump back to before step S70, and repeat the mapping process of the region in steps S70 to S110.

[0137] Following step S120, there is an optional step 140, in which the generated map and plan can be communicated to the fleet management server so that the status can be used for future operations or other vehicles. Alternatively, the method can proceed immediately from step S120 to step S150.

[0138] After step S120 or step S140, step S150 is performed, in which the load from the load pickup area is placed while precisely following... Figure 4B The generated trajectories are shown as examples in the attached figures, labeled 130, 132, and 134. Global attitude estimation is supported on the map, while utilizing the geometric properties identified in the loading area.

[0139] After placing the load in step S150, step S160 estimates whether the load is placed correctly. If this test in step S160 is negative, i.e. the load is not placed correctly or is stuck, then step S170 is performed to attempt to correct the load.

[0140] If the result of step S160 is positive, then in step S180, it is estimated whether the plan has been fully executed. If the plan has not been fully executed, the method allows the AGV to pick up the next load in the picking area before returning from step S180 to step S20.

[0141] If the planning is proven to be complete at the end of step S180, the AGV navigates to the waiting position in step S190 and the program ends in step S200.

[0142] Using this method, AGVs can efficiently and automatically transport loads from the pickup area to the loading area within the virtual boundary.

[0143] The controls that must be added to an existing automated guided vehicle (AGV) can be existing sensors that can be connected to the AGV or controls that can carry their own sensors, which can be added to existing AGVs or non-AGVs to enable the AGVs and non-AGVs to perform the methods of the present invention.

[0144] Alternatively, the operator for controlling the automated guided vehicle to transport the load from the load pickup area to the loading area on which the load will be placed may have all the means and equipment necessary to perform the method incorporated during manufacturing.

[0145] Figure 5A and Figure 5B The steps shown can also be used according to, for example Figure 6A and Figure 6B The vehicle used in the third implementation scheme shown is used for execution. Figure 6B The vehicle shown preferably includes a steering and drive mechanism for propelling and steering the AGV. Figure 6B In this configuration, the steering wheel and drive wheel are combined into a single wheel using passive casters on the side next to the steering / drive wheel. The passive wheels on the forks support the vehicle during transport loads (including when the forks are in the overhead position).

[0146] Other implementations may include independent steering wheels and drive wheels, as well as a pair of non-steering, separately controlled drive wheels with passively supported wheels (differential drive). The drive wheels and steering wheels are coupled to a guidance system consisting of sensors, a computing module, and a control interface, and are used to propel the AGV, steer the AGV, and move the lifting mechanism (preferably including a pair of forks). The sensors, computing module, and control interface are preferably... Figure 1B Those in it.

[0147] for Figure 5A and Figure 5B The method steps shown can take into account the following information: In this loading algorithm, it is assumed that the AGV has a map of the main operating area and is able to locate itself thereon before the loading operation begins. The "main operating area" refers to the area excluding truck / container space. According to... Figure 3 These zones are A+C, which are part of the general map possessed by the AGV, while zone B is unknown prior to the loading operation. Zone B is mapped during the loading operation, causing the general map to be expanded to include zone B. The understanding of "main operating area" differs from the term "operating area," which can also refer to the interior space of the truck / container.

[0148] The term "positioning" refers to the ability to calculate and continuously update the vehicle's current position and orientation (also known as attitude) relative to a zone map (global attitude estimation). The vehicle is also able to precisely follow a trajectory provided by the planning system. Trajectories constitute a comprehensive loading (and similar unloading) plan, consisting of individual trajectories for picking up and placing each individual load. Each trajectory is a set of consecutive attitudes to be followed to achieve a target attitude. For each trajectory (also called a plan or sub-plan), the guidance system calculates the required linear and angular velocities (or steering angle commands) to remain on the track, i.e., precisely follow the calculated and provided trajectory.

[0149] The vehicle's positioning system utilizes information from multiple sensors to calculate current attitude consistency. The primary (also called global) sensors are LIDARs 301A and 301B (or 2a, 2c of the first embodiment) in both directions of travel. The LIDARs are preferably 2D or 3D, but sensors 301A and 301B can be other optical or frequency technologies, such as cameras, sonar, or radar. LIDAR 301A is preferably mounted above the load. An optional LIDAR 305 (or 2b of the first embodiment) can be mounted on the side of the load-capturing mechanism below the load or, when close to other AGVs or human operation, on the side below the forks. LIDAR 301B is preferably a 2D or 3D LIDAR in the direction opposite to the direction of travel and can also be used as a safety sensor.

[0150] Reference numeral 305 preferably indicates a forward-facing safety LIDAR, which must have a complete (unobstructed) field of view in the direction of travel. Alternatively, a system such as... Figure 2 The image shows a pair of overlapping LIDARs. These sensors may be omitted if safety requirements are lower.

[0151] Local attitude estimation (in the vehicle coordinate system) is computed using IMU 303 and wheel and steering encoders 306, 307, and is mixed with global estimation to achieve consistent, drift-free global (in the map reference coordinate system) attitude estimation.

[0152] Upon receiving a loading task, the AGV follows to a known pick-up location to engage the load with a load-capturing mechanism, preferably a pair of forks, and the load preferably has fork slots. If the accuracy of the load pick-up location cannot be ensured or the dimensions cannot be conveyed, a camera 302 is used to profile the load and identify the fork slots. The AGV then lowers the lifting mechanism and engages the load. Once the load is captured, the lifting mechanism is raised to allow for further load transport and placement in a transport vehicle or container.

[0153] If no information about the transport is available (no internal space map) and no loading plan (loading pattern) has been generated, the AGV will attempt to enter the transport vehicle while maintaining a safe distance from the end of the wall or platform. During this initial, normal, slow entry process, a map update process is enabled, and observations from LIDAR 301A and camera 302 are incorporated into the main operational map. Camera 302 serves to verify load placement and can be located anywhere in the upper area to obtain the best view relative to the operational area. Camera 302 can be a single rangefinder camera (aka RGB-D or 3D), two cameras forming a stereo pair, or a single camera with an optional downward-facing distance sensor to obtain scale. Camera 302 can also be located in the same location as the upper LIDAR and / or tilting unit. Instead of a 3D camera, a tilting unit or a combination of a tilting unit and a 2D or 3D camera can be provided.

[0154] It may be necessary to travel several tens of centimeters to one or two meters inside the vehicle to accumulate sufficient information and expand the map, preferably several tens of centimeters after overcoming the loading ramps connecting the compartments / operable hall areas and the vehicle or container. During this step, geometric properties (also known as features), such as floor boundaries, walls, and their intersections, are also identified to aid in local attitude estimation within the vehicle.

[0155] After the map is expanded, virtual boundaries are added to the loading area, and a loading pattern (loading plan) is generated. The vehicle continues to move the load to the target location based on the generated loading trajectory. Once the target location is reached and it is verified that the AGV is below the distance threshold between adjacent loads or walls, the AGV lowers its lifting mechanism to place the load. The AGV then proceeds to the next load according to the loading plan, and so on, until the plan is complete.

[0156] During load placement operations, the required distance threshold may not be reached due to incorrect load configuration or other reasons. In addition to the drive motor current, or if a current sensor cannot be installed, the guidance system monitors a pressure sensor between the load and the vehicle to identify blockages caused by contact with other loads or walls. If the threshold is not reached, the vehicle is attempting to reach the target posture, but the current or pressure sensor is reporting a high increase in value, which is interpreted as a load jamming situation, and a correction operation is attempted.

[0157] Furthermore, if a loading plan cannot be generated during the map expansion step using a new transport vehicle due to an invalid map, unexpected objects in the transport vehicle, or a mismatch between the load and the transport dimensions in the required load capacity, the loading task is cancelled and the AGV is removed from transport. Depending on the error situation, manual intervention may be required, or the vehicle can attempt to repeat the transport area mapping process.

[0158] Independent of loading planning, including the first step of acquiring a map or the surroundings of the vehicle, the AGV can sense the surrounding obstacles and react accordingly based on the dynamics of these obstacles, replanning the path / trajectory of the surrounding static objects or waiting until the path clears the dynamic objects.

[0159] Therefore, it is safe for AGVs to enter a transport vehicle even without prior information about their interior area.

[0160] Compared to prior art document US 8,192,137 B2, the proposed invention preferably takes into account the pallet density of the maximum two rows per row relevant to forklift design. Using their proposed invention, the wall can be viewed and the vehicle profile analyzed before entering the vehicle to estimate the offset and angle relative to the intended transport arrangement at the gate.

[0161] This invention focuses on overcoming the limitations of existing technologies for single-pair forklift AGVs or other AGVs capable of transporting a single load but not equipped with a lateral movement mechanism, wherein the required load placement can be in any arrangement, including rows of two or more loads. This invention also solves the problem of load jamming during load placement, in which case existing technologies would be unable to identify the situation and would only place the load once the drive current or pressure increases.

[0162] Another significant difference between this invention and existing technologies is that it does not require directly identifying the offset and angle of the transport vehicle before it enters the transport vehicle. The natural existence of these parameters is merely a cause / trigger for the map expansion process steps and virtual boundary calculations, rather than the primary search target of the algorithm. While the presence of walls on the transport vehicle is advantageous for the loading method according to the invention, this is not a strict requirement, and the method can be applied to transport vehicles without walls at all, or to transport vehicles not placed at the gate, but, for example, to containers placed on the ground in the operating area.

[0163] The solution proposed in this invention may have an additional optical sensor that can sense and identify the contours of the platform, its dimensions, and the load placement in relation to adjacent load walls or virtual boundaries in the absence of walls.

[0164] Distance sensors mounted above the load in the fork direction are used to extend the map of the guidance system to new transport vehicles or containers and support tracking the AGV’s position and (importantly) orientation inside the transport vehicle during loading operations.

[0165] In the absence of walls in the vehicle, it may have limited auxiliary support, but by combining optical sensors with a second ranging sensor on the other side, it can perform position and orientation tracking tasks without difficulty.

[0166] A single guidance system exists for the entire operation based on natural navigation—a term well-known in the art—and requires no special external equipment installation or environmental modifications. The assisted position and orientation tracking is achieved by enabling auxiliary inputs in a multi-sensor hybrid approach, resulting in improved overall attitude estimation accuracy of the primary guidance system within the vehicle, without requiring switching between different methods.

[0167] Difficult situations may arise during entry into the transport vehicle, requiring the correction of positioning errors or ensuring they are minimized. In such cases, optional optical markers can be installed along the entry route, such as on the wall of a gate or other location, making them easily visible without needing to stop or cancel the AGV's route.

[0168] When an AGV does not have a side-shifting mechanism, the main problem in properly arranging loads in a transport vehicle is that, in order to place more than one load in a row (especially more than two loads) close to a side wall or another load, the AGV needs to turn and move toward the wall or load before turning and going straight again.

[0169] Typically, loads cannot be perfectly formed, and this operation may result in loads getting stuck or not placed close enough. Continuing the loading operation will lead to situations where not all planned loads can be placed inside the transport vehicle or container.

[0170] Therefore, the method for detecting such situations, verifying load placement, and having a correction step according to the present invention is helpful, and such a solution is also proposed in the present invention.

[0171] In summary, this invention enables the production of AGVs (Automated Guided Vehicles) specifically designed for transport vehicles or container loading. Furthermore, according to this invention, existing non-automated vehicles used for loading tasks can be automated by installing automation kits. In this way, sensors and computing modules are retrofitted into the vehicle's drive electronics to control speed, steer the wheels, and control the AGV's lifting mechanism.

[0172] The following will be referenced Figure 7 A fourth embodiment of the present invention is described. The fourth embodiment may utilize features of the first to third embodiments, and preferably utilizes features of the first embodiment, wherein the following additional considerations are taken into account:

[0173] To safely navigate and sense the environment around vehicle 201, ranging sensors 202a, 202b, and 202c and / or 202d (preferably distance sensors) are mounted in a manner that provides complete coverage in both directions of travel of vehicle 201. To cover situations where one sensor in each direction cannot deliver the required field of view due to occlusion, multiple overlapping sensors may be used, for example, when safety standards require it. The sensing technology will not affect the underlying control logic as long as it can provide similar or better accuracy to LiDAR. Alternatively, the sensors can be used precisely to ensure safe navigation and the required application accuracy. This means that a camera can also be used as a ranging sensor when the distance measurement or attitude estimation derived through image processing meets the above requirements.

[0174] Considering the minimum functional requirements of the fourth implementation scheme, the following sensors are needed:

[0175] Front and rear LiDAR

[0176] The front LIDAR (202c) is a LIDAR located on the other side of the fork pair, primarily used for rapid travel when unloaded and when carrying loads over long distances.

[0177] The rear LIDAR (202a) is located on the side of the fork pair above the load. It is primarily used for loading operations involving the load into trucks and containers.

[0178] The two LIDARs complement each other to improve the vehicle's attitude estimation in the operating area.

[0179] Both types of LIDARs can be 2D or 3D, or can be replaced by other sensors that can deliver distance data with similar quality to 2D or 3D LIDARs.

[0180] Where safety requirements are reduced or combined with alternative safety sensors or measures, a single 2D or 3D LiDAR or equivalent sensor is sufficient, provided that the sensor’s horizontal unobstructed field of view is greater than or equal to 180 degrees in the direction of the forks.

[0181] The IMU 204 is used in sensor fusion algorithms to improve vehicle attitude estimation in the environment.

[0182] When automating commercial vehicles, wheel encoders and steering encoders 203a and 203b are typically part of the system.

[0183] To ensure the proper load is present on the forks before lifting, load presence sensors 205a and 205b are installed. This can be a single sensor or a pair of sensors providing distance or capacity measurement, or a binary logic signal indicating the presence of a safe load.

[0184] To propel the vehicle, the guidance system on computing unit 206 calculates the required speed and conveys that required speed as a signal understandable to the speed controller of the automated vehicle. Therefore, the linear speed of the vehicle is controlled.

[0185] To ensure proper load / pallet placement on the forks or cargo box, an additional optical sensor or a pair of sensors 208a, 208b can be installed on the top. This sensor or sensor pair is installed in a manner that ensures a complete view of the load from the top while avoiding the aforementioned increase in maximum permissible height.

[0186] To support global attitude estimation for automated vehicles (i.e., in coordinates of the operational space (often referred to as the world coordinate system or global coordinate system), where attitude is related to position and orientation), additional sensors, such as wheel encoders 203a, 203b and IMU 204, are used. The information from these sensors 203a, 203b and 204, when combined, delivers a locally consistent attitude estimate, preferably in the robot's coordinate system, which can be transformed into the desired global frame. Sensors 203a and 203b are preferably wheel encoders integrated with the wheels (wheel and steering angle encoders or two wheel encoders, depending on the vehicle's kinematics). Alternatively, separate wheels with encoders can be provided attached to the vehicle body.

[0187] This attitude estimate drifts over time but is corrected by a global attitude estimate. Sensors 203a and 203b can be optical or magnetic encoders, preferably located on either side of vehicle 201 in the case of differential drive kinematics, or on the drive and steering motors in the case of tricycles or Ackerman kinematics. Sensor 204 is an IMU (Inertial Measurement Unit), which can be integrated into the computing module or located in another convenient location on vehicle 201.

[0188] The vehicle maintains its position and orientation, i.e., locates itself in the operating environment, by means of onboard ranging sensors 202a, 202b, 202c, 202d or equivalent sensors, optical or magnetic encoders 203a and 203b, and IMU 204. The IMU can be placed in any part of the vehicle, including the computing module.

[0189] The internal representation of the environment (map) is loaded from the server or obtained during the operation preparation process, which is typically performed once per new environment. The operation area consists of two parts—the known static area A (see...). Figure 8 ) and region B (which is unknown or not fully determined prior to this) Figure 8 ).

[0190] Figure 8 Zone B in the diagram represents the zone of the transport vehicle (container, trailer, or truck), where angle α indicates that, in this case, the container at the loading door of the trailer or container loading position is not strictly perpendicular to the wall / door of the gate. dx and dy represent the lateral and longitudinal displacements from the expected transport position. If certain characteristics of the transport vehicle (such as a list of expected dimensions) are known, these variables describe the uncertainty of the transport vehicle's exact location. The minimum requirement for the system to extend the operating area (map) into the unknown zone is the expected coordinates of the transport vehicle's entry point, such as the center position of the loading door, the expected entry coordinates of the container, etc. In other words, the vehicle (AGV / AMR) needs to reach the entry point location in a certain way to allow observation of the transport vehicle's internal space via installed onboard sensors. The displacements dx, dy, and α do not affect the mapping process.

[0191] At any time, if no match is found with prior known information, the area of ​​the transport vehicle can be mapped and an application can be made. Figure 4B The virtual boundary 110 in the diagram strictly defines the operational boundaries, even in the absence of one or more walls. The mapping process itself is a process of identifying the perimeter of the transport vehicle and its geometry in order to correctly assign the loading pattern. This process is typically performed once before the vehicle enters with the first load and updates the relevant navigation information, including pasting the identified area of ​​the transport vehicle onto the map. This applies to the first through fourth embodiments.

[0192] Now, we will explain what the process of identifying the perimeter means and how it relates to the process based on... Figure 8 The loading mode is associated with the generation.

[0193] Typically, a transportation system platform has a rectangular internal space for placing the transported load. In this example, the best way to define the perimeter of the vehicle is to find the four corners C1-C4, through which lines can be drawn to create a polygon defining the perimeter. This polygon is then added to the map of the pickup area, or at least to the map used to plan the pickup area that allows the vehicle to be fully positioned, planned, and navigated within the vehicle.

[0194] The four corners also allow for the complete determination of other geometric properties of the transport system, such as width, length, and angle α (e.g., if the transport vehicle is not directly docked at the loading gate). These parameters are useful for planning load placement patterns, where the found angle α can be used to rotate the pattern around a pivot point (which could be, for example, one of the four corners, such as C2) to properly adapt the pattern to the orientation of the transport system. Furthermore, the corners C1, C2 and C3, C4 that define the virtual left and right walls of the transport platform allow for the planning and maintenance of safe distances during loading and unloading operations.

[0195] In one example, the pattern is calculated relative to the top-left corner C2, and the derived orientation α is used to rotate the pattern around corner C2 to match the trailer orientation. In another example, any other corner from C1-C4 can be used.

[0196] The accuracy of the operation will depend on how precisely these corners are determined. Different filtering techniques can be used to ensure the correct location of corners and to reflect safe, traversable areas. The transport system may have different structures on the walls, or it may not have one or more walls. Through different ranging data processing methods or image data processing methods, for example, the walls or edges of the platform can be extracted and verified in terms of parallelism, or even if not fully observed, the intersections defining the corners can be found and further refined as more observations come in.

[0197] The loading and unloading operations of the fourth implementation scheme are then described in more detail.

[0198] The loading operation begins with a task order retrieved from a server or fleet management system. Other information includes the lost location, the quantity and size of the cargo / material to be loaded, the loading gate number, container location, or other transport entrance coordinates. Before placing the first load, the transport vehicle is automatically scanned to identify spatial dimensions, as mentioned above regarding corners C1, C2, C3, and C4, and optional angles α and dx, dy offsets. Based on the identified dimensions of the transport vehicle and the cargo / material dimensions and quantity, a loading pattern or plan, i.e., a complete loading plan, is generated. The loading plan is a list of sub-plans, preferably in the form of a trajectory, used to transport each individual load from the pick-up location to the appropriate location within the transport vehicle based on the generated loading pattern. Each sub-plan is a set of points describing the vehicle's target location and orientation, i.e., a set of postures to be reached sequentially. During the execution of each sub-plan, application-specific actions are defined, such as picking up, lowering the load, fully or partially raising or lowering the forks, etc. Each sub-plan ends with a lowering action. The overall planning and execution are managed by a task management algorithm, which reports any failures to the server or fleet management system and, if possible, performs recovery actions. Upon completion of the loading task, the vehicle reports success and navigates to the designated waiting location.

[0199] The placement of each load inside the transport vehicle is verified during and after each drop-off / placement operation using actuated 3D ranging or optical cameras 207a, 207b (single or dual depending on the type of load being transported), optional cameras 208a and 208b, and optionally using upper ranging sensors 202a and 202b, in single or dual configuration. If the load is incorrectly placed or there is a problem with load insertion, a correction is attempted. If correction fails, the loading task is paused, and the relevant fault is communicated to the server or the monitoring fleet management system. Manual correction can be attempted, after which the loading schedule can be resumed using the next load in the list.

[0200] This may also include solid-state LiDAR, 207a, 207b actuated 3D ranging, or optical cameras positioned to provide an optimal viewpoint of the carried load and adjacent loads during placement. Depending on the shape of the load, it may be necessary to extend the viewpoint, for example, by looking from the side to identify potential points of impact and to prevent the load from colliding, especially if the load is not properly shaped, tilted, or moved relative to the pallet it is placed on. Due to the specific nature of loading / unloading operations and because the task is performed very close to the vehicle walls, such cameras are not always in a fixed position at the vehicle. Therefore, actuated and / or optionally retractable cameras may be used, which can extend the viewpoint as needed and as possible, and can retract if they may be damaged.

[0201] During planning execution, the area along the travel direction is monitored relative to the presence of obstacles. If an obstacle appears in a potential collision zone, planning execution is paused. If the obstacle does not disappear within a defined time period and remains stationary, a replanning attempt is made. If replanning fails or the new plan (e.g., trajectory) cannot be accurately followed, the loading operation is stopped, and a failure is reported.

[0202] The unloading operation begins with a task order, received from a server or fleet management system, in a manner similar to the loading operation. This task order contains information about the unloading gate number or the location of the transport vehicle, as well as information about the load (including quantity, size, and placement location). The truck or container is scanned to (if applicable) find angle α and offset, and to verify internal space dimensions, or to identify the perimeter of the transport vehicle. The load is then scanned using overhead ranging sensors 202a, 202b, actuation sensors 207a and 207b, and optionally cameras 208a and 208b. A placement identification algorithm identifies the load placement based on ranging and / or additional image data, and compares the load placement with information received from the server. The vehicle then calculates an unloading plan, which, for example, consists of sub-plans constituting the transport trajectory and actions for each individual load, as described in the loading operation above.

[0203] exist Figure 8 In the diagram, areas A and B are shown as interchangeable pick (unload) areas and drop (load) areas.

[0204] Distance sensors mounted above the load in the fork direction are used to extend the map of the guidance system to the arriving transport vehicle and support tracking the AGV’s position and (importantly) orientation within the transport vehicle during loading operations, thereby allowing complex navigation operations and trajectory planning to be performed while taking into account all constraints of the transport space.

[0205] In the absence of walls in the vehicle, it may have limited auxiliary support, but by combining optical sensors with a second ranging sensor located on the other side, it can perform position and orientation tracking tasks without difficulty.

[0206] A single guidance system exists for the entire operation based on natural navigation—a term well-known in the art—and requires no special external equipment installation or environmental modifications. The optional auxiliary position and orientation tracking is achieved by enabling auxiliary inputs in a multi-sensor hybrid approach, resulting in improved overall attitude estimation accuracy of the main guidance system within the vehicle, without requiring switching between different methods. Otherwise, position and orientation tracking is indistinguishable from operation in the main area.

[0207] List of reference numerals

[0208] 1 vehicle

[0209] Sensors 2a, b, and c

[0210] Sensors 3a and 3b

[0211] Sensors 5a and 5b

[0212] Sensors 6a and 6b

[0213] 7 Sensors and Computing Units

[0214] 10 vehicles

[0215] 12c, d, e sensors

[0216] Field of view of rangefinders 14a, b, c, d, e

[0217] 16a-e Measurement Coverage

[0218] (A), (B), (C) areas

[0219] 30a, 30b Attitude synchronization markers

[0220] 32a, 32b Attitude synchronization markers

[0221] 100 Loading Area

[0222] 110 Virtual Boundary

[0223] 112 Front

[0224] 130 trajectory

[0225] 132 Trajectory

[0226] 134 Trajectory

[0227] Vehicle 201

[0228] Sensors 202a, b, c, and d

[0229] 203a and 203b sensors

[0230] 204 Sensors / IMU

[0231] 205a and 205b sensors

[0232] 206 computing units

[0233] 207a and 207b sensors

[0234] 208a and 208b sensors

[0235] 301A, B LIDAR

[0236] 302 camera

[0237] 303 IMU

[0238] 304 Computing Unit

[0239] 305 LIDAR

[0240] 306 and 307 Steering and Wheel Encoders

Claims

1. A method for automatically controlling a vehicle to transport at least two loads from a load pickup area where the at least two loads will be placed in corresponding loading areas to an operating area, wherein the method comprises the following steps: Obtain at least information about the pickup location, the quantity and size of the load to be transported; Scan the operating area to at least identify the spatial dimensions of the loading area; Generate a loading pattern for transporting the at least two loads from the load pickup area to the loading area, wherein the loading pattern includes the target locations and target orientations of the vehicles to be sequentially reached; The loading mode is executed until the loading task of transporting the at least two loads to the operating area is completed. Scanning the operation area includes: obtaining information about at least three corners of a defined polygon in which at least two loads will be placed, and optionally, adding the polygon of the operation area to the pick area before generating the loading pattern for transporting the at least two loads from the load pick area to the loading area. Obtaining information about the at least three corners of the defined polygon includes: using filtering techniques to determine a loading area that can be traversed with predefined precision, and wherein optionally ranging or image data processing is used to determine at least one wall of the polygon relative to the operating area, at least one of the at least two loads being placed on the at least one wall; Before generating the loading pattern, the two corners of the polygon are used to determine the tilt (α) between the operation area and the pickup area and optionally the two-dimensional offset (dx, dy) between the operation area and the pickup area.

2. The method of claim 1, wherein upon completion of the loading task, the vehicle reports successful execution of the loading task and navigates the vehicle to a predetermined waiting location.

3. The method of claim 1 or 2, wherein if a fault occurs during the execution of the loading mode, a recovery action to correct the fault is performed, and if the fault correction fails, the fault is reported to the server or fleet management system.

4. The method of claim 1 or 2, wherein the loading pattern comprises sub-plans in the form of a trajectory, wherein each sub-plan ends with a descent action with respect to the load.

5. A control system suitable for performing the method of any one of claims 1 to 4.

6. A vehicle capable of automatically transporting at least two loads from a load pickup area where the at least two loads will be placed in corresponding loading areas to an operating area, wherein the vehicle comprises: A means for obtaining at least information about the pickup location, the quantity and size of the load to be transported, and the loading area; A means for scanning the loading area to at least identify the spatial dimensions of the loading area; A means for generating a loading pattern for transporting the at least two loads from the load pickup area to the loading area, wherein the loading pattern includes the target locations and target orientations of the vehicles to be reached sequentially; as well as A means for executing the loading mode to automatically transport the at least two loads to the operating area until the loading task is completed. Scanning the operation area includes: obtaining information about at least three corners of a defined polygon in which at least two loads will be placed, and optionally, adding the polygon of the operation area to the pick area before generating the loading pattern for transporting the at least two loads from the load pick area to the loading area. Obtaining information about the at least three corners of the defined polygon includes: using filtering techniques to determine a loading area that can be traversed with predefined precision, and wherein optionally ranging or image data processing is used to determine at least one wall of the polygon relative to the operating area, at least one of the at least two loads being placed on the at least one wall; Before generating the loading pattern, the two corners of the polygon are used to determine the tilt (α) between the operation area and the pickup area and optionally the two-dimensional offset (dx, dy) between the operation area and the pickup area.

7. The vehicle of claim 6, wherein the means for performing the loading mode is capable of identifying problems of incorrect load placement or load insertion by using at least one 3D rangefinder or optical camera (207a, 207b).

8. The vehicle of claim 7, wherein the means for performing the loading mode is capable of attempting to correct the load placement in the event of incorrect load placement and problems with load insertion, and is configured to communicate the correction failure to a server or a monitoring fleet management system if the correction fails.

9. The vehicle of claim 7 or 8, wherein the at least one 3D ranging or optical camera (207a, 207b) is actuable or retractable to change the viewpoint of the load being carried and / or adjacent loads when performing the loading task.

Images