Pallet loading device and method using lidar

Abstract

A pallet loading device using LiDAR and a method thereof are disclosed. A pallet loading device using a lidar mounted on an unmanned forklift according to an embodiment of the present invention includes a lidar that emits a laser and converts range data reflected from a pin-shaped pallet into 3D point cloud data; a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; a pin recognition unit that performs channel normalization using the 2D BEV image as input data and recognizes the pin position through computation using a CNN (Convolutional Neural Network); a forklift controller that controls the operation of the unmanned forklift according to an applied control signal; and a control unit that identifies the difference between the pin position of a first pin-shaped pallet loaded on the fork of the unmanned forklift and the pin position of a fixed second pin-shaped pallet and applies a control signal to the forklift controller for correction to a matching position.

Description

Technology Field

[0001] The present invention relates to a pallet loading device and method using a lidar, and more specifically, to a pallet loading device and method using a lidar for automating multi-stage stacking of terminal-type pallets. Background Technology

[0002] In general, standard pallets are used in factories or industrial sites for the logistics transport of goods or parts. A standard pallet has multiple items loaded on its flat upper surface, and is lifted by inserting a forklift fork into the space / insertion hole formed in the lower part, and then transported to a desired location.

[0003] Meanwhile, recently, unmanned forklifts are being operated for logistics transfer within factories to realize Smart Factories, and pallet recognition technology is being developed for the automation of logistics transfer.

[0004] For example, in conventional pallet transfer automation technology using unmanned forklifts, a technology is being developed to recognize standard pallets using mono, stereo cameras and LiDAR and to estimate the position of the pallet by extracting feature lines from the top and bottom of the pallet.

[0005] However, conventional pallet recognition technology has the problem of being unable to recognize non-standard pallets and thus unable to automate multi-level stacking of non-standard pallets.

[0006] The matters described in this background technology section are written to enhance understanding of the background of the invention and may include matters that are not prior art already known to those skilled in the art to which this technology belongs. The problem to be solved

[0007] The embodiments of the present invention aim to provide a pallet loading device and method using LiDAR, which converts a pallet image using LiDAR into a BEV (Bird's eye view) to measure the position of the prongs configured on the pallet and controls an unmanned forklift for multi-stage stacking of prong-type pallets. means of solving the problem

[0008] According to one aspect of the present invention, a pallet loading device using a lidar mounted on an unmanned forklift according to an embodiment of the present invention comprises: a lidar that emits a laser and converts range data reflected from a pin-shaped pallet into 3D point cloud data; a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; a pin recognition unit that performs channel normalization on the 2D BEV image as input data and recognizes the pin position through computation using a CNN (Convolutional Neural Network); a forklift controller that controls the operation of the unmanned forklift according to an applied control signal; and a control unit that identifies the difference between the pin position of a first pin-shaped pallet loaded on the fork of the unmanned forklift and the pin position of a fixed second pin-shaped pallet and applies a control signal to the forklift controller for correction to a matching position.

[0009] Additionally, the pallet loading device may further include a communication unit that receives a transfer operation command from a server, the command including at least one of a pallet ID for transferring parts, a transfer destination, a travel path, and whether or not to load.

[0010] In addition, the lidar is installed on the lift device of the fork, and its position can be varied vertically according to the upward and downward movements of the fork.

[0011] In addition, the BEV conversion unit can generate the 2D BEV image by making the height (H) value smaller among the length (L), width (W), and height (H) of the 3D point cloud data.

[0012] In addition, the BEV conversion unit can generate a 2D BEV image of the upper or lower plate by reducing the height (H) value based on the upper plate or lower plate of the plate-shaped pallet.

[0013] In addition, the BEV conversion unit can convert the 2D image generated for each point value of the height (H) differently by passing it through a processor that projects it after coordinate translation, rotation, and scaling according to the angle of the lidar.

[0014] In addition, the above-mentioned snail recognition unit can display the image area recognized as the snail in the 2D BEV image of the rectangular border through image feature extraction and classification using the above-mentioned CNN, and identify the center point as the location of the snail.

[0015] In addition, the forklift controller measures the distance between the fork position and the fork and controls the unmanned forklift to lift the first fork-type pallet with the fork.

[0016] In addition, the forklift controller can temporarily store the position of the first forefoot pallet loaded on the fork by matching it to a motion coordinate system for attitude control of the fork, and can vary the position of the forefoot according to the motion of the unmanned forklift and the control of the fork's ascent or descent.

[0017] In addition, the control unit can measure the horizontal distance (d1), vertical distance (d2), and angle of deviation (θ) between the upper leg position of the second leg-shaped pallet and the lower leg position of the first leg-shaped pallet that corresponds to it.

[0018] In addition, the control unit can generate the control signal to correct the horizontal distance (d1), vertical distance (d2), and misaligned angle (θ) of the lower stem position based on the upper stem position.

[0019] In addition, the control unit can continuously calculate the positional difference of the horizontal distance (d1), vertical distance (d2), and tilted angle (θ), and if the set matching condition is not satisfied, apply a control signal to the forklift controller in the form of a closed-loop control.

[0020] In addition, when the control unit satisfies the alignment conditions in which the horizontal distance (d1) and vertical distance (d2) are 20mm or less and the angle of deviation (θ) is 2° or less, it finally transmits a fork lowering signal to the forklift controller so that the forklift can stack the forklift pallets in multiple stages.

[0021] Meanwhile, according to one aspect of the present invention, a method for a pallet loading device mounted on an unmanned forklift to load a pin-shaped pallet using LiDAR comprises: a) lifting a first pin-shaped pallet to be transported with the forks of the unmanned forklift and transporting it to a destination; b) emitting a laser using LiDAR to convert range data reflected from a fixed second pin-shaped pallet into 3D point cloud data; c) a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; and d) performing channel normalization on the 2D BEV image as input data and recognizing the pin position through computation using a CNN (Convolutional Neural Network). and e) a step of identifying the difference between the position of the first prong-shaped pallet loaded on the fork and the position of the second prong-shaped pallet fixed to the fork, and applying a control signal for correction to the matching position to a forklift controller that controls the operation of the unmanned forklift;

[0022] Additionally, step e) may include a step of calculating the horizontal distance (d1), vertical distance (d2), and angle (θ) that is twisted by the directional deviation between the upper leg position of the second leg-shaped pallet and the lower leg position of the first leg-shaped pallet that corresponds to it.

[0023] Additionally, the above step e) may include a step of continuously calculating the horizontal distance (d1), vertical distance (d2), and misaligned angle (θ), and if the alignment condition is not satisfied, controlling the unmanned forklift in a closed-loop control form to correct the position.

[0024] Additionally, step e) may include the step of transmitting a lowering control signal of the fork to the forklift controller to multi-stage stacking of the fork-type pallet when the horizontal distance (d1), vertical distance (d2) and the angle of deviation (θ) satisfy the alignment condition.

[0025] Additionally, prior to step a) above, the method may further include a step of collecting range data reflected from the first bell-shaped palette through the lidar, converting it into a 2D BEV image, and then recognizing the bell position through computation using the CNN.

[0026] Additionally, step a) above may include: a step of temporarily storing the position of the first forefoot-type pallet transferred to the fork by matching it to a motion coordinate system for attitude control of the fork of the forklift controller; and a step of varying the position of the forefoot according to the motion of the unmanned forklift and the upward or downward control of the fork.

[0027] Meanwhile, according to another aspect of the present invention, a server for controlling pallet loading of an unmanned forklift operated in a production plant comprises: a LiDAR signal collection unit that collects range data reflected from a pallet-shaped pallet from an infrastructure LiDAR arranged by area of ​​the production plant and converts it into 3D point cloud data; a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; a pallet recognition unit that performs channel normalization using the 2D BEV image as input data and recognizes the position of the pallet through computation using a CNN (Convolutional Neural Network); a forklift management unit that registers the ID of the unmanned forklift and collects status information of each unmanned forklift to monitor location tracking and operation status; and a transceiver unit that connects wirelessly with the unmanned forklift to collect the status information and transmits a control signal for the transfer and loading of the pallet-shaped pallet. and a central processing unit that identifies the difference between the position of the first leg of a pallet loaded on the fork of the unmanned forklift and the position of the second leg of a fixed pallet, and transmits a control signal to the unmanned forklift through the transmitting and receiving unit for correction to the matching position. Effects of the invention

[0028] According to an embodiment of the present invention, point cloud data of a pin-shaped pallet using LiDAR is converted into a BEV image, thereby enabling accurate recognition of the pin position.

[0029] In addition, the pallet's terminal position recognition function using LiDAR enables the autonomous logistics transfer and multi-level stacking of terminal-type pallets utilizing unmanned forklifts.

[0030] In addition, the implementation of a smart factory through the automation of logistics transport by unmanned forklifts can be expected to reduce factory operating costs and labor costs. Brief explanation of the drawing

[0031] FIG. 1 shows an unmanned forklift equipped with a pallet loading device using a lidar according to an embodiment of the present invention. Figure 2 shows the process of loading a pallet of a self-driving forklift according to an embodiment of the present invention. FIG. 3 is a block diagram schematically showing the configuration of a pallet loading device according to an embodiment of the present invention. Figure 4 shows the processing process of terminal-type pallet data measured in a lidar according to an embodiment of the present invention. FIG. 5 illustrates the process of converting 3D point cloud data into a 2D BEV image according to an embodiment of the present invention. FIG. 6 shows a state of multi-stage stacking of a terminal-type pallet according to an embodiment of the present invention. FIGS. 7 and 8 are flowcharts schematically illustrating a pallet loading method using a lidar according to an embodiment of the present invention. FIG. 9 shows a pallet loading system using a lidar according to another embodiment of the present invention. FIG. 10 is a block diagram schematically showing the configuration of a server according to another embodiment of the present invention. Specific details for implementing the invention

[0032] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0033] Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "…part," "…unit," and "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0034] Throughout the specification, terms such as first, second, A, B, (a), (b), etc., may be used to describe various components, but said components shall not be limited by said terms. These terms are intended only to distinguish a component from other components, and the nature, order, or sequence of said component is not limited by said terms.

[0035] Throughout the specification, when it is stated that one component is 'connected' or 'joined' to another component, it should be understood that it may be directly connected to or joined to the other component, or that there may be other components in between. Conversely, when it is stated that one component is 'directly connected' or 'directly joined' to another component, it should be understood that there are no other components in between.

[0036] Throughout the specification, the terms used are merely for describing specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0037] Throughout the specification, terms related to 'comprising,' 'having,' etc., are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0038] Unless otherwise defined in this specification, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as consistent with their meaning in the context of the relevant technology and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0039] Now, a pallet loading device and method using a lidar according to an embodiment of the present invention will be described in detail with reference to the drawings.

[0040] FIG. 1 shows an unmanned forklift equipped with a pallet loading device using a lidar according to an embodiment of the present invention.

[0041] Figure 2 shows the process of loading a pallet of a self-driving forklift according to an embodiment of the present invention.

[0042] Referring to FIGS. 1 and 2, an unmanned forklift (100) according to an embodiment of the present invention refers to an autonomous vehicle that automatically lifts and transports a pallet of the type of pin (hereinafter referred to as "pin pallet," 10) through a pallet loading device (110), a fork (120), and a lift device (130). In addition, in one aspect, the unmanned forklift (100) may refer to an autonomous mobile robot / automated guided vehicle (AMR / AGV) equipped with a fork (120) and a lift device (130) that freely transports and loads cargo.

[0043] The pallet loading device (110) plays a key role in automating the logistics transfer and loading of an unmanned forklift (100) based on the recognition of a terminal pallet (10) operated for logistics transfer within an automobile production plant using one or more lidars (111).

[0044] Vehicle production plants equipped with Smart Factories are utilizing terminal pallets for the transport of various parts.

[0045] A round-foot type pallet (10) includes a pallet (11) for carrying goods, a support (12) vertically positioned at each vertex of the pallet (11), and round feet (13) formed on the upper and lower parts of the support (12).

[0046] The bowl (13) has a concave groove formed like a bowl and has the shape of a structure that supports the pallet support (12), and is divided into an upper bowl (13a) and a lower bowl (13b) depending on the position where it is installed on the support (12).

[0047] The structure is stacked by positioning the lower end (13b) of a transfer target pallet to overlap with the upper end (13a) of a fixed end-type pallet (10) among a plurality of end-type pallets (10).

[0048] Hereinafter, for convenience of explanation, the pallet to be transported will be named the first terminal pallet (10-1) and the fixed pallet will be named the second terminal pallet (10-2).

[0049] That is, the unmanned forklift (100) can lift the first pallet (10-1) to be transported through the fork (120) and the lift device (130) and move the four lower pallets (13b) to a position corresponding to the four upper pallets (13a) of the fixed second pallet (10-2) to load them.

[0050] At this time, the unmanned forklift (100) can control the movement and lift device (130) by determining the relative position information of the first terminal pallet (10-1) loaded on the fork (120) and the second terminal pallet (10-2) fixed through the pallet loading device (110).

[0051] FIG. 3 is a block diagram schematically showing the configuration of a pallet loading device according to an embodiment of the present invention.

[0052] Referring to FIG. 3, a pallet loading device (110) according to an embodiment of the present invention includes a lidar (111), a BEV conversion unit (112), a terminal recognition unit (113), a forklift controller (114), a communication unit (115), and a control unit (116).

[0053] Lidar (111) emits a laser to the surroundings and measures the time and intensity of the reflected signal (light) to return, thereby detecting the distance, direction, speed, temperature, material distribution, and concentration characteristics of surrounding objects.

[0054] The lidar (111) is installed on the frame of the fork (120) and can be moved up and down without the need for a separate lifting device according to the operation of the fork (120).

[0055] Additionally, when the unmanned forklift (100) loads the first terminal pallet (10-1) onto the fork (120), the view of the lidar (111) may be obstructed. Therefore, the lidar (111) can be further installed at the lower central part of the lift device (130) to recognize the position of the fixed second terminal pallet (10-2).

[0056] Figure 4 shows the processing process of terminal-type pallet data measured in a lidar according to an embodiment of the present invention.

[0057] Referring to FIG. 4(A), the lidar (111) emits a laser to the surroundings and collects range data reflected from the round-shaped pallet (10), converting it into 3D point cloud data. The point cloud refers to a set cloud of multiple points spread out in three-dimensional space.

[0058] Referring to FIG. 4(B), the BEV conversion unit (112) converts the 3D point cloud data of the bird-shaped palette (10) into a 2D BEV (Bird's-Eye View) image through calculation.

[0059] For example, FIG. 5 illustrates the process of converting 3D point cloud data into a 2D BEV image according to an embodiment of the present invention.

[0060] Referring to Fig. 5, Equation 1 is a formula representing 3D point cloud data, and the grid of each point is represented by length (Length, L), width (Width, W), and height (Height, H).

[0061] The BEV conversion unit (112) generates a 2D BEV image by reducing the height (H) value among the length (L), width (W), and height (H) of the 3D point cloud data. The BEV conversion unit (112) can generate a 2D BEV image by reducing the height (H) value based on the upper pin (13a) or lower pin (13b) of the pin-shaped pallet (10) as needed.

[0062] At this time, the BEV conversion unit (112) can adjust the 2D image (x, y) generated for each point value of the height (H) to be converted differently by passing it through a processor that moves, rotates, and scales the coordinates according to the angle of the lidar and then projects it.

[0063] The fin recognition unit (113) performs channel normalization using the converted 2D BEV image as input data and recognizes the fin (13a / 13b) portion in the rectangular border image through image feature extraction and classification using a Convolutional Neural Network (CNN).

[0064] The fin recognition unit (113) displays an image area recognized as a fin in the 2D BEV image of the square border and can identify the center point (coordinates) as the location of the fin (see FIG. 4(B)).

[0065] In addition, the pallet recognition unit (113) stores various types of pallet models and design data including a pallet-type pallet (10), and supports the recognition and transfer of pallets through CNN-based image features and classification of LiDAR images.

[0066] The forklift controller (114) can control autonomous driving to a destination while detecting the surroundings by supplementing and integrating multiple sensing technologies through sensor fusion of at least one of radar, ultrasonic sensor, and camera as well as lidar (111).

[0067] The forklift controller (114) operates the lift device (130) according to the applied control signal to raise or lower the fork (120) and forms a motion coordinate system for posture control of the fork (120).

[0068] The forklift controller (114) measures the distance between the fork (120) and the position of the fork in the 2D BEV recognized through the fork recognition unit (113), and lifts the fork-type pallet (10) with the fork (120).

[0069] The forklift controller (114) can temporarily store the position of the first leg-shaped pallet (10-1) loaded on the fork (120) by matching it to the motion coordinate system and can vary the position of the leg according to the motion of the unmanned forklift (100) and the up / down control of the fork (120). Through this, the forklift controller (114) can load the first leg-shaped pallet (10-1) loaded on the fork (120) by varying the position of the lower leg (13b) to match the position of the upper leg (13a) recognized from the fixed second leg-shaped pallet (10-2).

[0070] In addition, the forklift controller (114) can control driving, steering, gear shifting, speed and braking, etc. for autonomous driving of the unmanned forklift (100).

[0071] The communication unit (115) transmits and receives data for the operation of the unmanned forklift (100) by linking with the production factory server (Manufacturing Execution System, MES) via wireless communication.

[0072] For example, the communication unit (115) can receive transfer operation commands from the server (MES), such as pallet ID, transfer destination, driving path, and loading status, for supplying parts by small-scale process or factory.

[0073] The control unit (116) is configured as a computing system that stores various programs and data for operating an unmanned forklift (100) in a production plant according to an embodiment of the present invention in memory and controls the overall operation of a pallet loading device (110) based thereon.

[0074] FIG. 6 shows a state of multi-stage stacking of a terminal-type pallet according to an embodiment of the present invention.

[0075] Referring to FIG. 6, the control unit (116) identifies the position of the lower end of the first end of the pallet (10-1) currently loaded on the fork (120).

[0076] The control unit (116) measures 3D point cloud data of a fixed second bell-shaped pallet (10-2) using a lidar (111), converts it into a 2D BEV image, and identifies the position of the upper bell (13a) in the projected image.

[0077] The control unit (116) measures the horizontal distance (d1), vertical distance (d2), and angle (θ) of deviation due to direction deviation between the position of the upper pin (13a) of the second pin-shaped pallet (10-2) and the position of the lower pin (13b) of the first pin-shaped pallet (10-1) that corresponds to each other. Through this, the control unit (116) can generate a control signal to correct the distance (d1, d2) and the angle (θ) of the lower pin (13b) position relative to the position of the upper pin (13a).

[0078] The control unit (116) controls the position correction behavior of the unmanned forklift (100) by applying a control signal to the forklift controller (114) so ​​that the position of the lower pin (13b) is aligned with the position of the upper pin (13a).

[0079] At this time, the control unit (116) can continuously calculate the positional difference between the horizontal distance (d1), vertical distance (d2), and angle of deviation (θ) between the two leg-shaped pallets, and if the set matching condition is not satisfied, transmit a control signal to the forklift controller (114) in the form of a closed-loop control.

[0080] Afterwards, when the control unit (116) satisfies the alignment conditions where the horizontal distance (d1) and vertical distance (d2) are 20mm or less and the angle of deviation (θ) is 2° or less, it finally transmits a fork lowering signal to the forklift controller (114) so ​​that the fork-type pallet (10) can be stacked in multiple stages.

[0081] For multi-stage stacking of such a terminal pallet (10), the control unit (116) may be implemented as one or more processors that operate according to a set program, and the set program may be programmed to perform each step of the pallet stacking method using a lidar according to an embodiment of the present invention.

[0082] Meanwhile, based on the aforementioned pallet loading device (110), a pallet loading method using a lidar according to an embodiment of the present invention is explained through the following Fig. 7.

[0083] In the preceding description, the configuration of the pallet loading device (110) was subdivided by function into parts / machines for the sake of understanding, but it is obvious that it can be integrated into a single pallet loading device (110). Therefore, in describing the pallet loading method using a LiDAR according to an embodiment of the present invention below, the pallet loading device (110) will be described as the subject of each step.

[0084] FIGS. 7 and 8 are flowcharts schematically illustrating a pallet loading method using a lidar according to an embodiment of the present invention.

[0085] Referring to FIGS. 7 and 8, a pallet loading device (110) according to an embodiment of the present invention controls an unmanned forklift (100) to autonomously drive to the destination of a first terminal pallet (10-1) to be transported, in accordance with a pallet transport command from a server (MES).

[0086] The pallet loading device (110) emits a laser using a lidar (111) and collects range data reflected from the first terminal pallet (10-1) and converts it into 3D point cloud data (S1).

[0087] The pallet loading device (110) converts the 3D point cloud data of the first terminal pallet (10-1) into a 2D BEV image through calculation (S2).

[0088] The pallet loading device (110) performs channel normalization using the 2D BEV image as input data and recognizes the lower pin (13b) in the rectangular border image of the 2D BEV image through computation using CNN (S3). At this time, the pallet loading device (110) can identify the location of the lower pin (13b) by extracting the center point of each of the four lower pins (13b).

[0089] The pallet loading device (110) aligns the unmanned forklift (100) based on the position of the lower fork (13b) and lifts the first fork-type pallet (10-1) with the fork (120) to transport it to the next destination (S4). At this time, the pallet loading device (110) can align the position of the unmanned forklift (100) by measuring the difference in distance and direction (angle) between the position of the lower fork (13b) and the fork (120) and transmitting a control signal to the forklift controller (114) to correct it.

[0090] The pallet loading device (110) collects range data reflected from the second round-shaped pallet (10-2) through the lidar (111), converts it into a 2D BEV image, and then recognizes the position of the upper round (13a) through computation using CNN (S5). This S5 step is similar to performing steps S1 through S3, except that the 2D BEV image is generated based on the upper round (13a).

[0091] The pallet loading device (110) calculates the horizontal distance (d1) and vertical distance (d2) and the angle (θ) that is twisted due to the directional deviation between the position of the upper leg (13a) of the second leg-shaped pallet (10-2) and the position of the lower leg (13b) of the first leg-shaped pallet (10-1) that corresponds to each other (S6).

[0092] The pallet loading device (110) controls the movement (position) of the unmanned forklift (100) by applying a control signal to the forklift controller (114) to align the position of the lower leg (13b) with the position of the upper leg (13a) (S7).

[0093] At this time, the pallet loading device (110) continuously calculates the horizontal distance (d1), vertical distance (d2) and the angle of deviation (θ), and if the alignment condition is not satisfied (S8; No), it can correct the position by controlling the unmanned forklift (100) in a closed-loop control form.

[0094] On the other hand, if the horizontal distance (d1), vertical distance (d2) and angle of deviation (θ) satisfy the alignment condition (S8; yes), the pallet loading device (110) lowers the fork (120) to stack multiple terminal pallets (10) in multiple stages (S9).

[0095] As such, according to an embodiment of the present invention, point cloud data of a pin-shaped pallet using LiDAR is converted into a BEV image, thereby providing the effect of accurately recognizing the pin position.

[0096] In addition, the pallet's terminal position recognition function using LiDAR enables the autonomous logistics transfer and multi-level stacking of terminal-type pallets utilizing unmanned forklifts.

[0097] In addition, the implementation of a smart factory through the automation of logistics transport by unmanned forklifts can be expected to reduce factory operating costs and labor costs.

[0098] Although embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and various other modifications are possible.

[0099] For example, in the above-described embodiment of the present invention, the pallet loading device (110) was described primarily in terms of recognizing a terminal pallet (10) using a lidar (111) mounted on an unmanned forklift (100). However, the embodiment of the present invention is not limited thereto, and a server operating infrastructure lidars arranged by area within a production plant can control the operating status of each unmanned forklift (100) based on terminal pallet location information calculated centrally.

[0100] For example, FIG. 9 shows a pallet loading system using a lidar according to another embodiment of the present invention.

[0101] FIG. 10 is a block diagram schematically showing the configuration of a server according to another embodiment of the present invention.

[0102] Referring to FIGS. 9 and 10, a pallet loading system according to another embodiment of the present invention includes an unmanned forklift (100), an infra-lidar (211), and a server (200). Since the only difference from the previous embodiment is that the pallet loading device is applied to the server (200), redundant descriptions are omitted and the differences are described in detail.

[0103] The unmanned forklift (100) includes a forklift controller (114) and a communication unit (115), and the lidar (111), BEV conversion unit (112), terminal recognition unit (113), and control unit (116) applied to the existing pallet loading device (110) are omitted.

[0104] Therefore, the forklift controller (114) can control the transfer and loading operation of the terminal pallet (10) according to the control signal received from the server (200) through the communication unit (115).

[0105] The infrastructure lidar (211) is fixedly positioned on the ceiling structure in each area of ​​the production plant and transmits the lidar signal received in the detection area to the server (200).

[0106] The server (200) includes a LiDAR signal collection unit (210), a BEV conversion unit (220), a forklift recognition unit (230), a forklift management unit (240), a transmission and reception unit (250), and a central processing unit (260).

[0107] The lidar signal collection unit (210) collects range data reflected from the infra-lidar (211) on the bell-shaped pallet (10) and converts it into 3D point cloud data.

[0108] The description of the BEV conversion unit (220) and the terminal recognition unit (230) is omitted as it can be understood by referring to the description of the preceding BEV conversion unit (112) and terminal recognition unit (113).

[0109] The forklift management department (240) registers the IDs of unmanned forklifts operating in the production plant and collects status information for each to track their location and monitor their operating status.

[0110] The transmitting and receiving unit (250) connects wirelessly with the unmanned forklift (100) to collect the above status information and transmits a control signal for transporting and loading the terminal pallet (10).

[0111] The central processing unit (260) stores various programs and data for controlling an unmanned forklift (100) operated in a production plant according to another embodiment of the present invention in a DB, and remotely controls the overall operation for loading pallets of the unmanned forklift (100) based thereon.

[0112] The central processing unit (260) searches for and selects the unmanned forklift (100) in the forklift management unit (240), and transmits a transfer operation command, such as pallet ID, transfer destination, driving path, and loading status, to operate it.

[0113] The central processing unit (260) identifies the difference between the position of the first terminal pallet (10-1) loaded on the fork (120) of the unmanned forklift (100) and the position of the second terminal pallet (10-2) in a fixed state, and transmits a control signal for correction to the matching position to the unmanned forklift (100) through the transmission and reception unit (250).

[0114] At this time, the central processing unit (260) measures the horizontal distance (d1) and vertical distance (d2) and the angle of inversion (θ) of the square frame between the position of the upper leg (13a) of the fixed second leg-shaped pallet (10-2) and the position of the lower leg (13b) of the first leg-shaped pallet (10-1) that corresponds to each other using an infra-lidar (211). Then, the control signal for correcting the difference between the horizontal distance (d1), vertical distance (d2), and angle of inversion (θ) is transmitted to the forklift controller (114), and when the alignment condition is satisfied, the first leg-shaped pallet (10-1) can be lowered to enable multi-stage stacking.

[0115] In addition, the central processing unit (260) can remotely perform the same control function as the control unit (116) in the preceding embodiment via a wireless network, and can centrally control and monitor pallet transfer and loading operations by unmanned forklift ID.

[0116] Thus, by omitting the configuration of the pallet loading device for each unmanned forklift, component costs can be reduced, and there is an advantage in that the operation of the unmanned forklift can be fully autonomous by processing recognition information of terminal-type pallets using infra-LiDAR at a central location.

[0117] The embodiments of the present invention are not limited to being implemented only through the apparatus and / or methods described above, but may also be implemented through a program for realizing a function corresponding to the configuration of the embodiments of the present invention, a recording medium on which the program is recorded, etc., and such implementation can be easily achieved by a person skilled in the art to which the present invention pertains from the description of the embodiments described above.

[0118] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention. Explanation of the symbols

[0119] 10: Bowl-shaped palette 11: Palette 12: Support 13: Bowl 13a: Upper stem 13b: Lower stem 100: Automated forklift 110: Pallet loading device 111: LiDAR 112: BEV converter 113: Leg recognition unit 114: Forklift controller 115: Communication unit 116: Control unit 120: Fork 130: Lift device 200: Server

Claims

Claim 1 A pallet loading device using a lidar mounted on an unmanned forklift comprises: a lidar that emits a laser and converts range data reflected from a pallet-shaped pallet into 3D point cloud data; a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; a pallet recognition unit that performs channel normalization using the 2D BEV image as input data and recognizes the position of the pallet through computation using a CNN (Convolutional Neural Network); and a forklift controller that controls the operation of the unmanned forklift according to an applied control signal. A pallet loading device using a LiDAR, comprising: a control unit that identifies the difference between the position of a first leg-type pallet loaded on the fork of the unmanned forklift and the position of a second leg-type pallet fixed thereon, and applies a control signal to the forklift controller for correction to a matching position; wherein the control unit measures the horizontal distance (d1), vertical distance (d2), and misalignment angle (θ) between the upper leg position of the second leg-type pallet and the lower leg position of the first leg-type pallet that corresponds to it. Claim 2 A pallet loading device using a lidar according to claim 1, further comprising a communication unit that receives a transfer operation command from a server including at least one of a pallet ID for transferring parts, a transfer destination, a driving path, and whether or not to load. Claim 3 A pallet loading device using a lidar installed in the lift device of the fork in accordance with the upward and downward movement of the fork, wherein the lidar is variable in position. Claim 4 In claim 1, the BEV conversion unit is a pallet loading device using a LiDAR that generates the 2D BEV image by making the height (H) value smaller among the length (L), width (W), and height (H) of the 3D point cloud data. Claim 5 In claim 4, the BEV conversion unit is a pallet loading device using a LiDAR that generates a 2D BEV image of the upper or lower leg of the upper leg or generates a 2D BEV image of the lower leg by reducing the height (H) value based on the upper leg or lower leg of the leg-shaped pallet. Claim 6 In claim 4, the BEV conversion unit is a pallet loading device using a lidar that converts the 2D image generated for each point value of the height (H) differently through a processor that projects the image after coordinate translation, rotation, and scaling according to the angle of the lidar. Claim 7 A pallet loading device using LiDAR according to claim 1, wherein the pin recognition unit displays an image area recognized as the pin in a 2D BEV image with a rectangular border through image feature extraction and classification using the CNN, and identifies the center point as the location of the pin. Claim 8 A pallet loading device using a lidar, wherein the forklift controller measures the distance between the fork position and the fork and controls the unmanned forklift to lift a first fork-type pallet with the fork. Claim 9 A pallet loading device using a lidar according to claim 1 or 8, wherein the forklift controller temporarily stores the position of the first leg of the pallet loaded on the fork by matching it to a motion coordinate system for attitude control of the fork, and varies the position of the leg according to the motion of the unmanned forklift and the raising or lowering control of the fork. Claim 10 delete Claim 11 A pallet loading device using a lidar, wherein the control unit generates a control signal to correct the horizontal distance (d1), vertical distance (d2), and misaligned angle (θ) of the lower terminal position based on the upper terminal position. Claim 12 A pallet loading device using a lidar according to claim 1 or 11, wherein the control unit continuously calculates the positional difference of the horizontal distance (d1), vertical distance (d2) and tilted angle (θ) and applies a control signal to the forklift controller in the form of a closed-loop control if the set matching condition is not satisfied. Claim 13 A pallet loading device using a lidar, wherein, in claim 12, the control unit satisfies the matching conditions in which the horizontal distance (d1) and vertical distance (d2) are 20mm or less and the twisted angle (θ) is 2° or less, and finally transmits a fork lowering signal to the forklift controller to multi-stage stack a terminal-type pallet. Claim 14 A method for a pallet loading device mounted on an unmanned forklift to load a pin-shaped pallet using LiDAR, comprising: a) a step of lifting a first pin-shaped pallet to be transported with the forks of the unmanned forklift and transporting it to a destination; b) a step of emitting a laser using LiDAR to convert range data reflected from a fixed second pin-shaped pallet into 3D point cloud data; c) a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; and d) a step of performing channel normalization on the 2D BEV image as input data and recognizing the pin position through computation using a CNN (Convolutional Neural Network). and e) a step of identifying the difference between the position of the first leg of a pallet loaded on the fork and the position of the second leg of a fixed pallet, and applying a control signal for correction to a matching position to a forklift controller that controls the operation of the unmanned forklift; wherein step e) includes a step of calculating the horizontal distance (d1), vertical distance (d2), and angle (θ) twisted by directional deviation between the upper leg of the second leg of the pallet and the lower leg of the first leg of the pallet that corresponds to each other. A pallet loading method using a lidar. Claim 15 delete Claim 16 In claim 14, the above step e) comprises a step of continuously calculating the horizontal distance (d1), vertical distance (d2), and misaligned angle (θ), and if the alignment condition is not satisfied, controlling the unmanned forklift in a closed-loop control form to correct the position, thereby providing a pallet loading method using LiDAR. Claim 17 A pallet loading method using a lidar according to claim 14 or 16, wherein step e) comprises the step of transmitting a lowering control signal of the fork to the forklift controller to multi-stage stacking of a terminal pallet when the horizontal distance (d1), vertical distance (d2) and twisted angle (θ) satisfy the alignment condition. Claim 18 A pallet loading method using a lidar according to claim 14, further comprising, prior to step a), a step of collecting range data reflected from the first bell-shaped pallet through the lidar, converting it into a 2D BEV image, and then recognizing the bell position through computation using the CNN. Claim 19 A pallet loading method using a LiDAR according to claim 14 or 18, wherein step a) includes: a step of temporarily storing the position of the first terminal pallet loaded on the fork by matching it to a motion coordinate system for attitude control of the fork of the forklift controller; and a step of varying the position of the terminal according to the motion of the unmanned forklift and the upward or downward control of the fork. Claim 20 A server for controlling pallet loading of an unmanned forklift operating in a production plant comprises: a LiDAR signal collection unit that collects range data reflected from a pin-shaped pallet from infrastructure LiDARs deployed in each area of ​​the production plant and converts it into 3D point cloud data; a BEV conversion unit that converts the 3D point cloud data into a 2D BEV (Bird's-Eye View) image through computation; a pin recognition unit that performs channel normalization using the 2D BEV image as input data and recognizes the pin position through computation using a CNN (Convolutional Neural Network); a forklift management unit that registers the ID of the unmanned forklift and collects status information of each unmanned forklift to monitor location tracking and operational status; and a transceiver unit that connects to the unmanned forklift via wireless communication to collect the status information and transmits control signals for the transfer and loading of the pin-shaped pallet. A server comprising: a central processing unit that identifies the difference between the position of a first leg-type pallet loaded on the fork of the unmanned forklift and the position of a fixed second leg-type pallet, and transmits a control signal to the unmanned forklift through the transmitting and receiving unit for correction to a matching position.

Images