Intelligent distribution vehicle and control method thereof

Abstract

The intelligent distribution vehicle control method comprises the following steps: identifying an object; generating 2D point cloud data based on the data output from the 2D sensor, and generating 3D point cloud data based on the data output from the 3D sensor; correcting an offset of the 3D point cloud data based on the 2D point cloud data; and determining a position of a stack support mounted on the object based on the corrected 3D point cloud data.

Description

Technical Field

[0001] The present disclosure relates to an intelligent delivery vehicle that can stably stack pallets and a method for controlling the intelligent delivery vehicle. Background Art

[0002] Smart delivery vehicles are being introduced into general warehouses and factories, as well as in smart factories that use individual components to manufacture items with different specifications, to ensure flexible and efficient supply and transportation of parts.

[0003] Smart delivery vehicles are a general concept that includes autonomous mobile robots (AMRs), automated guided vehicles (AGVs), and autonomous forklifts, and these smart delivery vehicles can move and perform tasks under the control of a control system.

[0004] The method of stacking pallets in multiple layers can be used in smart factories, etc. to efficiently use the space of the warehouse. To this end, the smart delivery vehicle can extract the characteristic lines of the upper and lower ends of the flat pallet, estimate the position of the pallet based on the extracted characteristic lines, and stack the loaded pallets at the estimated position of the pallet using a single camera, a stereo camera, or LiDAR.

[0005] Unlike flat pallets, in order for an intelligent delivery vehicle to stack column type pallets, it is important to accurately determine the positions of stacking supports mounted on the columns of the column type pallets.

[0006] The description provided above as a related art of the present disclosure is only for helping understanding the background of the present disclosure and should not be construed as being included in the related art known to those skilled in the art. Summary of the invention

[0007] [Technical issues]

[0008] An object of the present disclosure is to accurately determine the positions of stacking supports mounted on columns of a column type pallet by using 2D and 3D LiDAR sensors, and to stably stack column type pallets loaded on a vehicle onto the stacking supports.

[0009] The technical subject matter implemented in the present disclosure is not limited to the above-mentioned technical problems, and other technical subject matters not stated herein will be clearly understood by those skilled in the art from the following description.

[0010] [Technical solution]

[0011] To achieve the purpose of the present disclosure, a method for controlling an intelligent delivery vehicle according to an embodiment of the present disclosure may include: identifying a target; creating 2D point cloud data based on data output from a 2D sensor and creating 3D point cloud data based on data output from a 3D sensor; correcting an offset of the 3D point cloud data based on the 2D point cloud data; and determining a position of a stacking support installed on the target based on the corrected 3D point cloud data.

[0012] In addition, to achieve this purpose, the intelligent delivery vehicle according to an embodiment of the present disclosure may include: a sensing unit, including a 2D sensor and a 3D sensor configured to sense a target; a data processor, configured to create 2D point cloud data based on data output from the 2D sensor, create 3D point cloud data based on data output from the 3D sensor, and correct an offset of the 3D point cloud data based on the 2D point cloud data; and a position determiner, configured to determine a position of a stacking support installed on a target based on the corrected 3D point cloud data.

[0013] [Beneficial Effects]

[0014] According to various embodiments of the present disclosure described above, by accurately determining the positions of stacking supports mounted on the columns of the column type pallet via 2D and 3D LiDAR sensors, column type pallets loaded on a vehicle may be stably stacked on the stacking supports.

[0015] Effects obtainable by the present disclosure are not limited to the above-described effects, and other effects may be clearly understood by those skilled in the art from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram illustrating an example of a configuration of a smart factory applicable to an embodiment of the present disclosure.

[0017] Figure 2 is a block diagram showing an example of a configuration of a control system applicable to an embodiment of the present disclosure.

[0018] Figure 3 is a block diagram showing an example of a configuration of a smart delivery vehicle that may be applied to an embodiment of the present disclosure.

[0019] Figure 4 is a perspective view showing an example of the appearance of a smart delivery vehicle that can be applied to an embodiment of the present disclosure.

[0020] Figure 5 is a flow chart showing an example of a driving process of a smart delivery vehicle that can be applied to an embodiment of the present disclosure.

[0021] Figure 6is a block diagram showing an example of a configuration of a smart delivery vehicle according to an embodiment of the present disclosure.

[0022] Figure 7 is a perspective view showing an example of the appearance of a smart delivery vehicle according to an embodiment of the present disclosure.

[0023] Figure 8 2 is a view showing an example of a column-type pallet loaded onto / unloaded from a smart delivery vehicle according to an embodiment of the present disclosure.

[0024] Fig. 9 is a view showing an example of 3D point cloud data created from a 3D LiDAR sensor according to an embodiment of the present disclosure.

[0025] Fig.10 is a diagram illustrating an example of 2D point cloud data created from a 2D LiDAR sensor according to an embodiment of the present disclosure.

[0026] Fig.11 is a view showing an example of a process of recognizing a column of a column type pallet based on 2D and 3D point cloud data in an embodiment of the present disclosure.

[0027] Fig.12 1 is a view showing an example of a process of determining a distance and a rotation angle of stack supports between column type pallets in an embodiment of the present disclosure.

[0028] Fig.13 is a flowchart illustrating a method of controlling a smart delivery vehicle according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and regardless of the numbering of the accompanying drawings, the same or similar components are given the same reference numerals and are not described repeatedly. The terms "module" and "unit" used for components in the following description are only used for convenience of description without distinguishing meanings or functions. In the following description, if it is determined that the detailed description of the known technology related to the present disclosure makes the subject matter of the embodiments described herein unclear, the detailed description is omitted. In addition, the accompanying drawings are provided only for ease of understanding of the embodiments described herein without limiting the spirit described herein, and should be understood to include all changes, equivalents and substitutes within the spirit and scope of the present disclosure.

[0030] Terms including ordinal numbers such as "first" and "second" may be used to describe various components, but these components should not be construed as being limited to these terms. These terms are only used to distinguish one component from another.

[0031] It should be understood that when an element is referred to as being “connected to” or “coupled to” another element, it may be directly connected to or directly coupled to the other element, or connected to or coupled to the other element with other elements interposed therebetween. On the other hand, it should be understood that when an element is referred to as being “directly connected to” or “directly coupled to” another element, it may be connected to or coupled to the other element without other elements interposed therebetween.

[0032] Unless the context clearly indicates otherwise, singular forms are intended to include plural forms.

[0033] It should be further understood that the terms “including” or “having” used in this specification specify the presence of stated features, steps, operations, components, parts or a combination thereof, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or a combination thereof.

[0034] The unit or control unit included in the name of the internal configuration of the intelligent delivery vehicle or control system is only a term commonly used to name the controller that controls a specific function, rather than referring to a general functional unit. For example, each controller may include a modem / transceiver that communicates with another controller or sensor to control the corresponding function, a memory that stores an operating system or logic commands and input / output information, and one or more processors that perform determinations, calculations, decisions, etc. for controlling the corresponding function. Depending on the implementation, one processor may be responsible for the calculations of multiple controllers.

[0035] First, refer to Figure 1 Describe the configuration of a smart factory in which smart delivery vehicles according to implementation schemes are deployed and employed.

[0036] Figure 1 is a block diagram showing an example of a configuration of a smart factory applicable to an embodiment of the present disclosure.

[0037] See also Figure 1 , the smart factory 100 may include a smart delivery vehicle 110 , a manufacturing system 120 , a monitoring system 130 , and a control system 140 .

[0038] According to the manufacturing process and the target manufacturing speed, the smart factory 100 may include a plurality of smart delivery vehicles 110, a plurality of manufacturing systems 120, and a plurality of monitoring systems 130. Hereinafter, the components are described.

[0039] First, the intelligent delivery vehicles may include autonomous mobile robots (hereinafter, for convenience, referred to as "AMR"), automatic guided vehicles (hereinafter, for convenience, referred to as "AGV"), and automatic forklifts. Depending on the adoption strategy of the intelligent delivery vehicles 110 in the smart factory 100, only one of the AGV or AMR may be used, and both the AGV and the AMR may operate in a single smart factory 100.

[0040] The AGV performs the required operations (movement, rotation, stop, etc.) in the smart factory 100 by identifying and following the guidance facilities installed on the floor to guide the AGV. The guidance facilities may represent optically recognizable marks (dots, 2D codes, etc.), tags that can be identified in a short-distance non-contact type (e.g., NFC tags, RFID tags, etc.), magnetic strips, lines, etc., but these are examples, and the present disclosure is not necessarily limited to this. The guidance facilities can be arranged continuously or discontinuously spaced apart from each other on the floor. The AGV requires that the guidance facilities have been installed in advance before use, because they basically perform operations by identifying and following the guidance facilities, so when the AGV needs to be moved to a new path or an existing path needs to be changed, new facilities need to be installed or existing guidance facilities need to be physically changed. In addition, since the AGV does not leave the path set by the guidance facilities, when an obstacle is sensed on or around the path, the AGV usually stops until the object is removed or they are specifically controlled. In order to adopt AGV, the control system 140 should control the AGV based on the guidance facilities, so the control system 140 can send instructions to the AGV at the current position, such as "drive until the third mark is recognized", "turn 90 degrees when the third mark is recognized", etc., in units of individual instructions or in units of tasks including multiple instructions (for example, collection, supply, charging, patrolling, etc.).

[0041] The AMR can determine the current position (i.e., positioning) by sensing the surrounding environment, and the AMR can plan a path by itself (path planning) by positioning and using a map, which can be considered the most distinguishable point from the AGV. Thus, when a map providing compatible coordinates is shared between the AMR and the control system 140, the control system 140 can control the AMR by giving a path to the AMR based on the coordinates. In addition, when an obstacle is sensed during driving, the ARM can return after avoiding the obstacle by setting an avoidance path by itself. The function of the control system 140 to set one or more waypoint coordinates as the path of the AMR can be referred to as global path planning, and the function of the AMR to set a moving path or an avoidance path between the waypoint coordinates determined by the global path planning can be referred to as local path planning.

[0042] See below Figure 3 and Figure 4 A more detailed configuration of the intelligent delivery vehicle 110 is described and will be referred to in Figure 5Describe the driving control process of AMR.

[0043] The manufacturing system 120 may refer to a system (e.g., a robot arm, a conveyor belt, etc.) that performs a manufacturing process of a product in the smart factory 100, and in a broader sense, may refer to a system that is arranged to help perform tasks (such as the entry and exit of the smart delivery vehicle 110) when the manufacturing process is performed by a person. The system for assisting in performing a task may be a system for monitoring the state of a designated location (at which the smart delivery vehicle 110 may drop a pallet carried thereon or may pick up a pallet in an area where a specific manufacturing process is performed), a system for determining a process state, a system for managing entry and exit in an area, etc., but the system is not limited thereto.

[0044] For example, manufacturing system 120 may be controlled by a programmable logic controller (PLC) and may communicate with control system 140 in connection with the progress of a process.

[0045] The monitoring system 130 may perform a function of obtaining information for determining a situation in the smart factory 100 and transmitting the information to the control system 140. For example, the monitoring system 130 may include a camera, a proximity sensor, etc., but is not limited thereto.

[0046] The control system 140 communicates with the above-mentioned components 110, 120, and 130, so as to be able to obtain information for adopting the smart factory 100 or to be able to control the components. For example, the control system 140 can deploy the smart delivery vehicle 110, set a path, assign tasks, manage the process of each item, manage materials, etc.

[0047] In an embodiment, the control system 140 may include: a local control system (AMR / AGV control system (ACS)), which controls the surrounding processing facilities based on the position of the AGV / AMR and controls the AGV / AMR based on the task; and an integrated control system (mobile robot integrated monitoring system (MoRIMS)), which controls two or more local control systems in an integrated manner. The integrated control system (MoRIMS) can control the status and path of all intelligent distribution vehicles 110 in the smart factory 100, set the distribution process, and cooperate with multiple local control systems to control traffic. For example, when the local control system (ACS) is provided for intelligent distribution robots of the same manufacturer or the same type, the integrated control system (MoRIMS) can perform integrated control for preventing collisions based on information obtained through multiple local control systems, such as analysis of bottleneck levels in intersection / overlapping areas, acceleration / deceleration control during driving, and reconstruction of avoidance paths through traffic distribution control between different types.

[0048] In addition, the integrated control system (MoRIMS) can also have a manufacturing execution system (MES) as an upper-level control body, and the MES can be similar to an automatic scheduler (Advanced Planning and Scheduling (APS)).

[0049] In addition to the above-mentioned components 110, 120, 130 and 140 of the smart factory 100, devices for communication between components (such as access points (APs)), chargers for charging the smart delivery vehicles 110, loading spaces for storing or loading components, spaces for keeping finished products or intermediate products, traffic signals, gates, spare spaces for parking smart delivery vehicles 110, etc. may also be appropriately arranged in the smart factory 100.

[0050] Thereafter, see Figure 2 The configuration of the control system 140 applicable to the embodiment of the present disclosure is described.

[0051] Figure 2 is a block diagram showing an example of a configuration of a control system applicable to an embodiment of the present disclosure. Figure 2 The components shown in are components related to the embodiments of the present disclosure, and more or fewer components may be included to actually implement the control system 140.

[0052] See also Figure 2 The control system 140 may include a firmware management unit 141 , a traffic control unit 142 , a process control unit 143 , a manufacturing / distribution management unit 144 , an inventory management unit 145 , a communication unit 146 , a vehicle monitoring unit 147 , and a map management unit 148 .

[0053] The firmware management unit 141 obtains the latest firmware of the smart delivery vehicle 110 through the communication unit 146 and sends the latest firmware to the smart delivery vehicle 110, so that the firmware is updated, thereby keeping the firmware of the smart delivery vehicle 110 up to date.

[0054] Traffic control unit 142 may control traffic signals and gates based on the path of smart delivery vehicle 110 , and may also recalculate the path of smart delivery vehicle 110 according to traffic.

[0055] The process control unit 143 can determine a process for each article, and can manage tasks such as the progress of the process and the process position.

[0056] The manufacturing / distribution management unit 144 can deploy smart delivery vehicles 110 based on the mission.

[0057] The inventory management unit 145 manages the location and quantity of materials, and this information can be used to use the process more efficiently, for example, by launching a smart delivery vehicle 110 to the destination before the point in time when the materials are actually assembled / consumed to pick up or collect the pallet.

[0058] The communication unit 146 can communicate not only with the internal components of the smart factory 100 (such as the smart delivery vehicle 110, the manufacturing system 120 and the monitoring system 130), but also with other objects (such as a firmware update server).

[0059] The vehicle monitoring unit 147 can monitor the location, path, battery status, communication status, powertrain status, etc. of each smart delivery vehicle 110. In this case, the path is a concept including a global path based on waypoints and a real-time local path. In addition, the battery status may include voltage, current, temperature, peak values ​​of voltage and current, state of charge (SOC), state of health (SOH), etc. The communication status may include information about the currently activated communication protocol (Wi-Fi, etc.), the connected AP, the distance to the AP, the channel being used, etc. The powertrain status may include load, temperature, RPM, etc. in the drive system.

[0060] In addition, the vehicle monitoring unit 147 can also check the currently assigned tasks, operating modes, firmware versions, etc. of each smart delivery vehicle 110.

[0061] The map management unit 148 may obtain map data of a grid map type, which the AMR of the smart delivery vehicle 110 obtains when traveling in the smart factory 100, and may provide the factory manager with a tool that enables the factory manager to edit the obtained map data. The area, virtual lane, intersection, prohibited entry area, etc., in which the smart delivery vehicle 110 performs one or more preset operations when entering the area may be set by editing the map data, but this is an example and the present disclosure is not necessarily limited thereto. The map management unit 148 may distribute the map to smart delivery vehicles 110 other than the smart delivery vehicle 110 that initially obtained the grid map by actual driving through the communication unit 146.

[0062] Then, see Figure 3 and Figure 4 Describe the smart delivery vehicle.

[0063] Figure 3 is a block diagram showing an example of a configuration of a smart delivery vehicle that can be applied to an embodiment of the present disclosure.

[0064] See also Figure 3, the smart delivery vehicle 110 may include a vehicle portion 111, a sensing unit 112, a loading portion 113, a communication unit 114, and a controller 115. Hereinafter, the components are described.

[0065] The vehicle unit 111 may include a driving source, wheels, suspension, etc. that are involved in the movement, steering, and stopping of the intelligent delivery vehicle 110. An electric motor that supplies power from a built-in battery (not shown) may be used as a driving source. The wheels may include one or more driving wheels that are supplied with a driving force from a driving source and a non-driving wheel that rotates through the movement of the vehicle body but is not supplied with a driving force. According to an embodiment, when a plurality of driving wheels are provided, the driving source may be matched with each driving wheel, and the rotation of the driving wheel may be independently controlled. In this case, even without a specific steering system, steering may be performed by steering the vehicle body by making the rotation directions of different driving wheels different. At least some of the non-driving wheels may be caster wheels, but this is an example and the present disclosure is not necessarily limited thereto.

[0066] The sensing unit 112 for sensing the environment of the smart delivery vehicle 110, the operating status of the vehicle body, etc. may include at least one of a 2D laser scanner (e.g., LiDAR), a 3D vision (stereo) camera, a multi-axis gyroscope sensor, an acceleration sensor, a wheel encoder, and a proximity sensor.

[0067] The encoder may output information that makes it possible to determine how much the wheel rotates using light emitted from a light emitting device (e.g., a photodiode). For example, the encoder may count the number of slits provided in a circumferential direction on the wheel or a disk that rotates with the wheel in a unit time. The controller 115 is able to perform odometry of estimated displacement by analyzing the position amount that changes over time using data obtained by the encoder and the gyro sensor. However, the displacement estimated based on the encoder data may be different from the actual displacement due to slip or wear of the wheel (change in the dynamic radius of the wheel). Therefore, when performing odometry, the controller 115 can output a result close to the actual value by performing correction of noise and errors using information collected from the wheel and the gyro sensor through a predetermined algorithm (e.g., an extended Kalman filter (EKF)). This odometry method can be particularly useful when it is impossible to determine the current position (positioning is impossible) using a 2D laser scanner to be described below.

[0068] A 2D laser scanner can scan the surrounding environment by emitting laser light to the surrounding environment through a rotating mirror and sensing the return signal after reflection. In this case, the result of sensing the point cloud shape can be output by analyzing the intensity of the reflected signal and the time difference between emission and reception.

[0069] The 3D vision camera can calculate the distance to the object based on the time difference between two cameras spaced a predetermined distance from each other (i.e., the pixel distance between images respectively captured by the cameras). The texture projector emitting infrared light having a predetermined pattern can be set to be able to sense even flat objects having the same color (e.g., a white wall).

[0070] Typically, 2D laser scanners may be used for mapping, navigation, object recognition, etc., and 3D cameras may be particularly useful for avoiding objects in navigation, but this is an example and the present disclosure is not necessarily limited thereto.

[0071] The loading part 113 as a component for carrying the items to be loaded can be the top plate itself on the vehicle body, a table arranged on the top plate, a lift, a rotating table rotating on a vertical axis, a forklift, a conveying mechanism or a combination thereof. The forklift can support telescopic and tilting functions similar to ordinary forklifts.

[0072] The communication unit 114 can communicate with other components in the smart factory 100 (such as the manufacturing system 120 and the control system 140), support uniform communication between smart delivery vehicles 110, and even communicate with chargers when performing charging tasks.

[0073] The controller 115 , which is an object of overall control of the above-mentioned components 111 , 112 , 113 , and 114 , may determine a current task, a current position and a destination, plan a path, control a loading section, and the like based on information obtained from the control system 140 through the communication unit 114 .

[0074] Figure 4 is a perspective view showing an example of the appearance of a smart delivery vehicle that can be applied to an embodiment of the present disclosure.

[0075] See also Figure 4 , an exemplary AMR is shown as a smart delivery vehicle 110. The vehicle body as a whole may have a track-type flat shape having a long axis extending in a first axial direction. One drive wheel 111-1 may be arranged in the central portion of the vehicle body in the first axial direction and may be arranged on one side in the second axial direction, and another drive wheel (not shown) may be arranged in the second axial direction on the other side opposite to the one drive wheel 111-1. This arrangement of drive wheels may be referred to as "differential drive (DD)". Although in Figure 4Not shown in the figure, two or more non-driven wheels may be arranged under the vehicle body. In this case, when the two driving wheels rotate in the same direction at the same speed, the vehicle can move forward or backward in the first axial direction, and when they rotate in opposite directions at the same speed, the vehicle can rotate around a rotation axis extending in a third axial direction and passing through the center C of the vehicle body plane. The sensor unit 112 may be arranged on the front of the vehicle body and the loading portion 113 may be arranged on the top of the vehicle body. The loading portion 113 may be configured to be movable up and down in the third axial direction, and the bracket or tray may be fixed on the top by a guide 113-1.

[0076] However, the above Figure 4 The configuration of the AMR shown in is an embodiment, and it is obvious that the AGV may have a similar configuration or the AMR may have another configuration.

[0077] Next, refer to Figure 5 Describe the driving process of the intelligent delivery vehicle 110.

[0078] Figure 5 is a flow chart showing an example of a driving process of the smart delivery vehicle 100 that can be applied to an embodiment of the present disclosure. Figure 5 In the present invention, it is assumed that the intelligent delivery vehicle 110 is an AMR capable of performing positioning and local path setting for convenience.

[0079] See also Figure 5 ,First, the AMR can obtain the actually surveyed grid map (S501) through LiDAR and the like when driving in the smart factory 100.

[0080] When the AMR sends the obtained grid map to the control system 140, the editing and matching of the grid map can be performed in the map management unit 148 of the control system 140 (S502). The editing can include the process of setting the above-mentioned various areas on the grid map and the process of assigning costs to each grid. The closer to the obstacle or the non-entry area, the higher the cost can be assigned to prevent the AMR from moving around the obstacle or moving to an area that the AMR should not enter. This is because when setting a local path, the AMR selects the set of cells with the lowest cost between waypoints as the path.

[0081] In addition, map matching may refer to the process of matching coordinates between a CAD map used to design the smart factory 100, an actually surveyed map (LiDAR map), and an edited topology map.

[0082] Thereafter, the control system 140 may share the topology map with all AMRs in the factory through the communication unit 146 ( S503 ).

[0083] The other following steps may be a process applied to a single AMR.

[0084] The AMR may determine the current position (positioning) on ​​the map using the sensor data of the sensing unit 112 and the obtained map (S504). For example, the AMR may determine the current position by comparing the surrounding terrain features obtained by LiDAR with a map based on feature points.

[0085] The control system 140 may select a particular AMR and assign a task to the AMR, and the task may generally be assigned one or more waypoints determined by global path planning. Waypoints may be defined by coordinates on a map and may be accompanied by information about the direction in which the AMR is believed to be moving at the coordinates (i.e., the heading). As such a task is assigned, a destination may be set in the AMR (yes in S505), and the AMR may perform local path planning between waypoints based on the cost on the topological map (S506).

[0086] When the path is determined, the AMR starts traveling (S507), and when an obstacle is sensed by the sensing unit 112 during traveling (Yes in S508), the AMR can perform an evasive maneuver by searching for a local path for avoiding the sensed obstacle. In some cases, the control system 140 can change the mission of the AMR according to the evasive maneuver or the failure of the evasive maneuver.

[0087] Further, the AMR may correct the position error in movement by the above-mentioned mileage technology before driving until reaching the destination (S510).

[0088] Thereafter, when arriving at the destination (S511), the AMR may perform task-based maneuvers (S512). For example, the AMR may determine whether the conditions for entering a specific processing area have been cleared, or collect an empty tray at the destination, or unload an object loaded on the loading section 113.

[0089] Embodiments of the present disclosure provide an intelligent delivery vehicle that can stably stack column-type pallets loaded on the vehicle onto stacking supports by accurately determining positions of stacking supports mounted on the pallets.

[0090] The pallet may be a column type pallet and the stacking support may be a cup kit mounted on the column type pallet. In this case, the smart dispensing vehicle may stably stack the column type pallet by accurately determining the position of the cup kit mounted on the column of the column type pallet through 2D and 3D LiDAR sensors. In addition, the column type pallet may refer to a pallet having columns extending vertically at the corners of a common flat pallet and having cup kits respectively arranged at the upper and lower ends of each column and helping to maintain the stacking state. The following will refer to Figure 8Describe the shape in more detail.

[0091] In the following, reference Figure 6 A smart delivery vehicle according to an embodiment is described.

[0092] Figure 6 is a block diagram showing an example of a configuration of a smart delivery vehicle according to an embodiment of the present disclosure.

[0093] See also Figure 6 , the smart delivery vehicle 110a may include a vehicle portion 111, a sensing unit 112, a loading portion 113, and a controller 115, and the controller 115 may include a data processor 201, a position determiner 202, a loading controller 203, and a matching determiner 204. Hereinafter, the components are described.

[0094] The sensing unit 112 may include at least one 2D LiDAR sensor, a 3D LiDAR sensor, and a visual sensor for sensing a target object (column-type pallet) around the smart delivery vehicle 110a. The 2D LiDAR sensor may scan the column of the column-type pallet into a 2D shape, and the 3D LiDAR sensor may trace the column-type pallet into a 3D shape by emitting lasers in 3 dimensions. The 3D LiDAR sensor may generally monitor the shape of the column-type pallet, and the 2D LiDAR sensor may sense the position of the column of the column-type pallet more accurately than the 3D LiDAR sensor. The visual sensor may be implemented as an RGB image sensor.

[0095] The loading portion 113 may be implemented as a forklift loading object (column-type pallet) to be transported.

[0096] Hereinafter, for the convenience of description, the columnar pallet positioned around the smart delivery vehicle 110a is referred to as the "first pallet", and the columnar pallet loaded on the loading portion 113 of the smart delivery vehicle 110a is referred to as the "second pallet".

[0097] The data processor 201 may output point cloud data for determining the position of the stacking supports mounted on the columns of the first pallet based on data output from the 2D LiDAR sensor, the 3D LiDAR sensor, and the vision sensor.

[0098] First, the data processor 201 may create 2D point cloud data based on data output from the 2D LiDAR sensor, and may create 3D point cloud data based on data output from the 3D LiDAR sensor. In addition, the data processor 201 may identify the first pallet through a preset recognition algorithm based on data output from the visual sensor. When the first pallet is not identified, the data processor 201 may keep receiving data from the sensing unit 112.

[0099] When the first pallet is identified based on the data output from the visual sensor, the data processor 201 may pre-process the 2D point cloud data and the 3D point cloud data to remove noise from the data. In addition, the data processor 201 may transform the coordinates of the position of the first data shown in the pre-processed 2D point cloud data and the 3D point cloud data based on the position of the smart delivery vehicle 110a. Therefore, the 2D and 3D point cloud data may each include the coordinates of the position of the first pallet based on the position of the smart delivery vehicle 110a.

[0100] Since the 2D LiDAR sensor senses the position of the column of the column type pallet more accurately than the 3D LiDAR sensor, the data processor 201 may correct the offset of the 3D point cloud data based on the 2D point cloud data. In more detail, referring to Equation 1, the data processor 201 may determine the coordinates a, b, c of the column of the first pallet based on the 2D point cloud data, and may correct the coordinates of the column of the first pallet shown in the 3D point cloud data based on the coordinates a, b, c of the column of the first pallet at a preset radius R from the x-axis, y-axis, and z-axis.

[0101] Equation 1: (xa) 2 +(yc) 2 +(zc) 2 =R 2

[0102] The position determiner 202 may determine the position of the stacking support mounted on the column of the first pallet based on the corrected 3D point cloud data. In more detail, the position determiner 202 may identify the column of the first pallet based on the 3D point cloud data, and may calculate the coordinates of the position of the stacking support mounted on the column of the first pallet based on the recognition result. In addition, the position determiner 202 may determine the distance and rotation angle (degree of distortion) between the stacking support of the first pallet and the loading portion 113 based on the position of the stacking support of the first pallet and the position of the smart delivery vehicle 110a.

[0103] The loading controller 203 controls the movement and rotation of the vehicle portion 111 and the lifting and displacement of the loading portion 113 based on the distance and rotation angle between the stack support of the first pallet and the loading portion, thereby being able to control the position of the loading portion 113 on which the second pallet is loaded.

[0104] The matching determiner 204 may calculate the distance and the rotation angle of the stack support of the first tray and the stack support of the second tray, and may determine whether the stack positions match based on the calculation result.

[0105] When it is determined that the stacking position does not match, the loading controller 203 controls the movement and rotation of the vehicle part 111 and the lifting and displacement of the loading part 113 based on the distance and rotation angle between the stacking supports until the stacking position matches, thereby being able to control the position of the loading part 113 loaded with the second pallet.

[0106] When it is determined that the stack positions match, the loading controller 203 may perform control so that the second tray loaded on the loading portion 113 is stacked on the stack support of the first tray.

[0107] Figure 7 is a perspective view showing an example of the appearance of a smart delivery vehicle according to an embodiment of the present disclosure.

[0108] See also Figure 7 , an exemplary automatic forklift is shown as an intelligent delivery vehicle 110a. The vehicle body as a whole may have the following shape: having a long axis extending in a first axial direction. Wheels 111-1a and 111-2a may be arranged on one side of the vehicle body in a second axial direction, and other wheels (not shown) may be arranged on the other side of the vehicle body in the second axial direction to be opposite to wheels 111-1a and 111-2a. Fork 113a is arranged on the front of the vehicle body in the first axial direction and is capable of performing operations for displacing and lifting the load. In addition, on fork 113a, a rod-shaped mechanical switch (not shown) for maintaining a consistent load position of the load may be installed. 3D LiDAR sensors 112-1a and 112-2a and visual sensors 112-3a and 112-4a may be fixed by a sensor fixing member (not shown) mounted on fork 113a. The 3D LiDAR sensors 112-1a and 112-2a and the visual sensors 112-3a and 112-4a may move with the sensor fixing member according to the displacement and lifting of the fork 113a. The 3D LiDAR sensor 112-1a and the visual sensor 112-3a may be arranged on the left side of the center of the fork 113a, and the LiDAR sensor 112-2a and the visual sensor 112-4a may be arranged on the right side of the center of the fork 113a. One 2D LiDAR sensor 112-5a may be arranged at the central portion of one side of the vehicle body in the second axial direction, and another 2D LiDAR sensor ( Fig.10 112-6a) in the vehicle body may be arranged at a central portion of the other side of the vehicle body in the second axial direction to be opposite to the 2D LiDAR sensor 112-5a.

[0109] However, the above Figure 7 The configuration of the automatic forklift shown in is an example and may have other configurations.

[0110] Figure 82 is a view showing an example of a column-type pallet loaded onto / unloaded from a smart delivery vehicle according to an embodiment of the present disclosure.

[0111] See also Figure 8 , the column type pallet 220 may include column supports L1 to L4, columns P1 to P4 respectively connected to the column supports L1 to L4, stacking supports C1 to C4 respectively installed on the upper ends of the columns P1 to P4, and a body B respectively connected to the lower ends of the columns P1 to P4 at the corners. The column supports L1 to L4 and the stacking supports C1 to C4 may be implemented as a cup kit. Therefore, the smart delivery vehicle 110a unloads the column supports L1 to L4 of one column type pallet loaded on the loading part 113 onto the stacking supports C1 to C4 installed on another column type pallet, so that the column type pallets can be stacked in multiple layers.

[0112] However, the above Figure 8 The configuration of the column type pallet shown in FIG. 1 is an embodiment and may have other configurations.

[0113] Fig. 9 is a view showing an example of 3D point cloud data created from a 3D LiDAR sensor according to an embodiment of the present disclosure.

[0114] See also Fig. 9 , the 3D point cloud data shows the overall shape of the stacking supports C1 to C4 including the column type tray. The 3D LiDAR sensors 112-1a and 112-2a each have a field of view (FOV) and can estimate the positions of the stacking supports C1 to C4 using the 3D point cloud data in the FOV area.

[0115] Fig.10 is a diagram illustrating an example of 2D point cloud data created from a 2D LiDAR sensor according to an embodiment of the present disclosure.

[0116] See also Fig.10 , the 2D LiDAR sensor 112-6a may be disposed at the center portion of the other side of the vehicle body to Figure 7 The 2D LiDAR sensors 112-5a and 112-6a each have a FOV, and the 2D point cloud data shows the column of the column-type pallet as a 2D shape.

[0117] Fig.11 is a view showing an example of a process of recognizing a column of a column type pallet based on 2D and 3D point cloud data in an embodiment of the present disclosure.

[0118] Reference Fig.11, the “2D” area is the result of identifying the column of the column type pallet based on the 2D point cloud data created from the 2D LiDAR sensors 112-5a and 112-6a. The “3D” area is the result of identifying the column of the column type pallet based on the 3D point cloud data created from the 3D LiDAR sensors 112-1a and 112-2a. Therefore, the position determiner 202 can accurately determine the position of the stacking support mounted on the column of the column type pallet based on the recognition results in the “2D” and “3D” areas.

[0119] Fig.12 1 is a view showing an example of a process of determining a distance and a rotation angle of stack supports between column type pallets in an embodiment of the present disclosure.

[0120] See also Fig.12 , stacking supports C1 to C4 are mounted on the columns of the first pallet located around the intelligent traveling vehicle 110a, and stacking supports C1' to C4' are mounted on the columns of the second pallet loaded on the loading portion 113 of the intelligent traveling vehicle 110a. The stacking supports C1 to C4 correspond to the stacking supports C1' to C4', respectively. The matching determiner 204 may calculate the horizontal distance d1, the vertical distance d2, and the rotation angle θ between the stacking supports C1 to C4 of the first pallet and the stacking supports C1' to C4' of the second pallet.

[0121] Fig.13 is a flowchart illustrating a method of controlling a smart delivery vehicle according to an embodiment of the present disclosure.

[0122] Reference Fig.13 , the 2D LiDAR sensors 112-5a and 112-6a, the 3D LiDAR sensors 112-1a and 112-2a, and the visual sensors 112-3a and 112-4a of the sensing unit 112 can monitor the surrounding environment of the smart delivery vehicle 110a (S101). In this case, the data processor 201 can create 2D point cloud data based on the data output from the 2D LiDAR sensors 112-5a and 112-6a, and can create 3D point cloud data based on the data output from the 3D LiDAR sensors 112-1a and 112-2a.

[0123] The data processor 201 may determine whether the first tray has been recognized based on the data output from the visual sensors 112-3a and 112-4a (S103). When the first tray has not been recognized (No in S103), the sensing unit 112 may keep monitoring the environment (S101).

[0124] When the first pallet has been identified (yes in S103), the data processor 201 may pre-process the 2D point cloud data and the 3D point cloud data (S105), and may transform the coordinates of the position of the first pallet shown in the pre-processed 2D point cloud data and the 3D point cloud data based on the position of the intelligent delivery vehicle 110a (S107).

[0125] Thereafter, the data processor 201 may correct the offset of the 3D point cloud data based on the 2D point cloud data (S109). As described above, the data processor 201 may determine the coordinates of the column of the first pallet based on the 2D point cloud data, and may correct the coordinates of the column of the first pallet shown in the 3D point cloud data based on the coordinates of the column of the first pallet with a preset radius R from the x-axis, the x-axis, and the z-axis.

[0126] The position determiner 202 may determine the position of the column of the first pallet based on the corrected 3D point cloud data, and may determine the position of the stack supporter mounted on the column of the first pallet based on the recognition result ( S111 ).

[0127] When determining the position of the stacking support of the first pallet, the position determiner 202 can determine the distance and rotation angle between the stacking support of the first pallet and the loading portion 113 (S113), and the loading controller 203 can control the position of the loading portion 113 based on the distance and rotation angle between the stacking support of the first pallet and the loading portion 113 (S115).

[0128] The matching determiner 204 may calculate the distance and rotation angle between the stacking support of the first tray and the stacking support of the second tray loaded on the loading portion 113, and may determine whether the stacking positions match based on the calculation result (S117). When it is determined that the stacking positions do not match (No in S117), the loading controller 203 may control the position of the loading portion 113 based on the distance and rotation angle between the stacking supports (S115).

[0129] When it is determined that the stack positions match (Yes in S117 ), the loading controller 203 may perform control so that the second tray loaded on the loading portion 113 is stacked on the stack support of the first tray.

[0130] Meanwhile, the present disclosure can be implemented as computer-readable code on a program coding medium. Computer-readable media include all kinds of recording devices that maintain data that can be read by a computer system. For example, a computer-readable medium can be a HDD (hard disk drive), an SSD (solid state disk), an SDD (silicon disk drive), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. Thus, the detailed description should not be interpreted as being limited in all aspects, and should be interpreted as an example. The scope of the present disclosure should be determined by a reasonable analysis of the claims, and all changes within the equivalent scope of the present disclosure are included within the scope of the present disclosure.

[0131] [Reference Number Description]

[0132] 100: Smart Factory 110: Smart Delivery Vehicle

[0133] 120: Manufacturing system 130: Monitoring system

[0134] 140: Control system.

Claims

1. A method for controlling an intelligent delivery vehicle, the method comprising: Identify the target; creating 2D point cloud data based on data output from the 2D sensor and creating 3D point cloud data based on data output from the 3D sensor; Correcting the offset of the 3D point cloud data based on the 2D point cloud data; as well as A position of a stack support mounted on the target is determined based on the corrected 3D point cloud data.

2. The method according to claim 1, wherein: The identification includes: Monitoring the surrounding environment through visual sensors; and identifying the target based on data output from the visual sensor, and The method further includes pre-processing the 2D point cloud data and the 3D point cloud data when the target is identified.

3. The method according to claim 1, wherein: Both the 2D point cloud data and the 3D point cloud data include the coordinates of the location of the target based on the location of the smart delivery vehicle.

4. The method according to claim 1, wherein: The correction includes: Determining the coordinates of the column of the target based on the 2D point cloud data; and Based on the coordinates of the column of the object, the coordinates of the object shown in the 3D point cloud data are corrected by a preset radius.

5. The method according to claim 1, wherein: The determination includes: identifying a column of the target based on the corrected 3D point cloud data; and The coordinates of the position of the stack support mounted on the column of the target are determined based on the recognition result.

6. The method according to claim 1, further comprising: Based on the stack supporter mounted on the target, a position of a loading portion on which a load object is loaded is controlled.

7. The method according to claim 6, further comprising: The distance and the rotation angle between the stacking support of the target and the stacking support of the object are determined, and whether the stacking positions match is determined based on the determination result.

8. The method according to claim 7, wherein: When it is determined that the stack positions do not match, control of the position is performed so that the position of the loading portion is controlled based on the distance and the rotation angle between the stack supports.

9. The method according to claim 7, further comprising: When it is determined that the stacking positions match, control is performed so that the carried object is stacked on the stacking support of the target.

10. A computer-readable recording medium having a program for executing the method for controlling an intelligent delivery vehicle according to any one of claims 1 to 9.

11. An intelligent delivery vehicle, comprising: a sensing unit including a 2D sensor and a 3D sensor configured to sense an object; a data processor configured to create 2D point cloud data based on data output from the 2D sensor, create 3D point cloud data based on data output from the 3D sensor, and correct a shift of the 3D point cloud data based on the 2D point cloud data; as well as A position determiner is configured to determine a position of a stacking support mounted on the target based on the corrected 3D point cloud data.

12. The intelligent delivery vehicle according to claim 11, wherein: The sensing unit further includes a visual sensor configured to sense the target, and The data processor is further configured to recognize the target based on the data output from the visual sensor, and pre-process the 2D point cloud data and the 3D point cloud data when the target is recognized.

13. The intelligent delivery vehicle according to claim 11, wherein: Both the 2D point cloud data and the 3D point cloud data include the coordinates of the location of the target based on the location of the smart delivery vehicle.

14. The intelligent delivery vehicle according to claim 11, wherein: The data processor is further configured to determine the coordinates of the column of the target based on the 2D point cloud data, and correct the coordinates of the target shown in the 3D point cloud data by a preset radius based on the coordinates of the column of the target.

15. The intelligent delivery vehicle according to claim 11, wherein: The position determiner is further configured to recognize a column of the target based on the corrected 3D point cloud data, and determine coordinates of a position of the stack support mounted on the column of the target based on a result of the recognition.

16. The intelligent delivery vehicle according to claim 11, further comprising: A loading controller is configured to control a position of a loading portion on which a load object is loaded based on a stack supporter mounted on the target.

17. The intelligent delivery vehicle according to claim 16, further comprising: A match determiner is configured to determine a distance and a rotation angle between the stack support of the target and the stack support of the carrying object, and determine whether the stacking positions match based on the determination result.

18. The intelligent delivery vehicle according to claim 17, wherein: When the stacking positions do not match, the loading controller controls the position of the loading portion based on the distance and the rotation angle between the stacking supports.

19. The intelligent delivery vehicle according to claim 17, wherein: When the stacking positions match, the loading controller performs control so that the carried object is stacked on the stacking support of the target.

Images