The caterpillar-type agricultural transport robot capable of following LiDAR-based workers

Abstract

The present invention relates to a tracked agricultural transport robot capable of LiDAR-based worker tracking. The tracked agricultural transport robot according to the present invention is designed to transport crops harvested in open fields or agricultural materials used therein, and is capable of LiDAR-based worker tracking.

Description

Technology Field

[0001] The present invention relates to a tracked agricultural transport robot capable of LiDAR-based worker tracking.

[0002] Specifically, the present invention relates to a tracked agricultural transport robot designed to transport crops harvested in open fields or agricultural materials used therein, and capable of LiDAR-based worker tracking. Background Technology

[0003] Agriculture is an industry essential for human survival as it produces food, but it is also the slowest industry to innovate due to the persistence of traditional agricultural techniques. Furthermore, anxiety regarding future food production is increasing due to the decline and aging of the agricultural workforce. To address this, there are various attempts to integrate agriculture with technologies related to the tertiary and quaternary industries, one of which is the demand for the development of agricultural robot technology.

[0004] Agricultural robots are utilized in a variety of farming tasks, ranging from directly handling crops such as sowing, grafting, harvesting, and transportation, to monitoring the farming process and supporting crop cultivation through weeding, pest control, and the application of fertilizers and compost.

[0005] Meanwhile, agricultural robots currently under development feature structures suitable for specific farming tasks. Among them, open-field farming robots are utilized for large-scale planting, harvesting, and transportation, contributing to increased agricultural productivity. Furthermore, due to their multi-purpose utility, significant demand is predicted for them in terms of improving economic efficiency.

[0006] In particular, due to the nature of open-field operations, open-field agricultural robots require the essential integration of advanced ICT technologies such as GPS, IoT, and big data with operational control techniques that utilize a high level of awareness of driving conditions and the recognition of objects and crops. Furthermore, given that the primary users of agricultural robots are the elderly, technological development must consider ease of use as well as the convenience of fault diagnosis and maintenance. Prior art literature

[0007] Republic of Korea Published Patent Application 10-2022-0088991 The problem to be solved

[0008] The present invention provides a LiDAR-based tracked agricultural transport robot capable of worker following, which operates in any one of a worker following mode, an autonomous driving mode, and a manual control mode.

[0009] In particular, the present invention provides a LiDAR-based tracked agricultural transport robot capable of tracking a worker according to a setting mode by generating worker identification recognition data through a multi-object tracking algorithm using data acquired from a LiDAR sensor and a vision sensor.

[0010] In particular, the present invention provides a LiDAR-based tracked agricultural transport robot capable of worker following and autonomous driving based on a set path according to a set mode by performing driving path learning through the application of a Simultaneous Localization And Map-Building (SLAM) algorithm to data acquired from a LiDAR sensor. means of solving the problem

[0011] The present invention, devised to solve the above problem, relates to a tracked agricultural transport robot that operates in any one of the following modes: a worker-following mode in which it drives and steers by following the movements of a worker, an autonomous driving mode in which it drives and steers by following a preset path, and a manual control mode in which it drives and steers under the control of a worker via a remote controller.

[0012] The above-described tracked agricultural transport robot comprises: a loading section including a predetermined loading space formed by a guide designed to be positioned vertically in the height direction to prevent the load from falling out; a seesaw lifting section located below the loading section and designed to move the height of the loading section upward; a tracked drive section including a plurality of track rollers driven by receiving power from a drive motor, a track shoe that rotates on an orbit by the plurality of track rollers, and a roller frame that fixes the track rollers; a sensor section including a LiDAR sensor that acquires surrounding shape information and 3D point cloud data of surrounding objects according to driving and steering, a vision sensor that acquires surrounding shape information and RGB-D data of surrounding objects according to driving and steering, a GPS sensor that acquires real-time position according to driving and steering, and an inertial measurement device that acquires tilt data in the yaw, roll, and pitch directions in real time; and a worker classification recognition data generation section that generates worker classification recognition data through a Multiple Object Tracking (MOT) algorithm using 3D point cloud data acquired by the LiDAR sensor and RGB-D data acquired by the vision sensor. A driving path learning unit that performs driving path learning by applying a Simultaneous Localization and Map-Building (SLAM) algorithm to 3D point cloud data acquired by the above LiDAR sensor to generate a path map including a set path and estimating the driving path of a robot within the path map; a memory unit that stores worker classification recognition data generated by the above worker classification recognition data generation unit and a path map including a set path generated by the driving path learning unit; and a communication unit that receives a setting signal for any one of a worker following mode, an autonomous driving mode, and a manual control mode as a setting signal provided by a worker through the remote controller.and a control unit that controls the crawler-type drive unit to follow a worker based on the worker identification recognition data stored in the memory unit when the setting signal received from the communication unit is in a worker-following mode, controls the crawler-type drive unit to follow a set path within the path map stored in the memory unit when the setting signal received from the communication unit is in an autonomous driving mode, and controls the crawler-type drive unit to drive and steer according to a drive control signal provided by the worker via the remote controller when the setting signal received from the communication unit is in a manual control mode;

[0013] In one example, the seesaw lifting unit may include a scissor lift consisting of 1 to 3 stages; and a hydraulic cylinder that provides pressure to raise or lower the scissor lift.

[0014] In one example, the worker classification recognition data generation unit can generate worker classification recognition data by individually extracting feature points from the 3D point cloud data and RGB-D data, performing coordinate system transformation, and then applying the data obtained through calibration using a transformation matrix and an inner / outer matrix to any one of the Multiple Object Tracking (MOT) algorithms selected from Deep-SORT, AB3DMOT, UCMCTrack, and Joint Detection and Embedding.

[0015] In one example, the worker classification recognition data generation unit can generate worker classification recognition data by applying a Multiple Object Tracking (MOT) algorithm based on a detection-based tracking technique, which involves individually extracting feature points from the 3D point cloud data and RGB-D data, performing coordinate system transformation, applying a YOLO model to the data obtained through calibration using a transformation matrix and an inner / outer matrix to perform object detection, applying a convolutional neural network model to extract features, and then applying a Kalman filter and a Hungarian algorithm to perform object tracking.

[0016] In one example, the driving path learning unit may apply a Simultaneous Localization And Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to 3D point cloud data acquired by the LiDAR sensor, and generate a path map including a set path with accuracy correction performed by applying the SLAM algorithm using tilt data in the yaw, roll, and pitch directions acquired by the inertial measurement device, and perform driving path learning to estimate the driving path of the robot within the path map. Effects of the invention

[0017] The tracked agricultural transport robot according to the present invention operates in one of a worker-following mode, an autonomous driving mode, and a manual control mode according to the worker's selection, and has the advantage of enabling the implementation of various driving modes depending on the work situation or environment.

[0018] In particular, the tracked agricultural transport robot according to the present invention can drive and steer by following a worker based on a Multiple Object Tracking (MOT) algorithm using data acquired through multiple sensing according to a setting mode, or drive and steer by following a set path acquired through SLAM algorithm-based driving path learning, thereby facilitating open-field work and thus providing excellent effects in improving agricultural productivity.

[0019] Of course, the effects of the present invention are not limited to the scope mentioned above. Brief explanation of the drawing

[0020] FIG. 1 is a block-mixed perspective view for specifically explaining the structure and components of an endless-track agricultural transport robot according to the present invention. FIG. 2 is a diagram illustrating a worker-following mode in which an endless track agricultural transport robot according to the present invention drives by following a worker. FIG. 3 is a diagram illustrating a path-following mode in which an endless track agricultural transport robot according to the present invention drives and steers by following a set path. FIG. 4 is a side view illustrating the detailed structure of the tracked drive unit of a tracked agricultural transport robot according to the present invention. FIG. 5 is a block diagram illustrating the detailed configuration of the sensor unit of an endless track type agricultural transport robot according to the present invention. FIG. 6 is a block diagram for specifically explaining a worker classification recognition data generation algorithm performed in a worker classification recognition data generation unit of an endless track type agricultural transport robot according to the present invention. FIG. 7 is a diagram for specifically explaining the driving path learning performed in the driving path learning unit of an endless track type agricultural transport robot according to the present invention. Specific details for implementing the invention

[0021] Hereinafter, the present invention will be explained in more detail with reference to drawings and examples.

[0022] In this specification, singular expressions include plural expressions unless otherwise specified.

[0023] The terms used in this specification have been selected based on currently widely used general terms, taking into account their functions in the present invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0024] In this specification, terms such as "first," "second," etc., may be used to describe various components, but the components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0025] In this specification, terms such as “comprising” or “consisting” are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0026] In this specification, terms such as “to be made up” or “being made up” are intended to emphasize that no other configuration is included other than the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as excluding the possibility of the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof other than said configurations.

[0027] The ‘part’ presented in this specification performs at least one function or operation, and its operation may be controlled by at least one processor.

[0028] The present invention relates to a tracked agricultural transport robot.

[0029] The tracked agricultural transport robot according to the present invention is an agricultural machine used for harvesting crops and transporting agricultural materials, and operates in any one of the following modes: a worker-following mode in which it drives and steers by following the movements of a worker, an autonomous driving mode in which it drives and steers by following a set path obtained through driving path learning, and a manual control mode in which it drives and steers under manual control by a worker via a remote controller.

[0030] In the above, the term 'worker following mode' refers to a mode of driving and steering to follow a worker based on worker classification recognition data generated by the worker classification recognition data generation unit (500) and stored in the memory unit (700), as illustrated in FIG. 2.

[0031] In the above, the term 'autonomous driving mode' refers to a mode of driving and steering by following a set path within a path map that includes a set path obtained according to the learning result in the driving path learning unit (600) and stored in the memory unit (700), as illustrated in FIG. 3.

[0032] In the above, the term 'manual control mode' refers to a mode of driving and steering by receiving control signals from an operator provided through a remote controller, without operator tracking or path tracking.

[0033] The tracked agricultural transport robot according to the present invention has a technical feature of improving the degree of freedom of the work method according to the work situation or environment by following the operator according to the operator's selection, autonomously driving by following a learned set path, or driving and steering under the manual control of the operator.

[0034] The tracked agricultural transport robot according to the present invention also has another technical feature in that, in driving and steering that follow a worker, it follows a worker based on worker identification recognition data generated by applying a Multiple Object Tracking (MOT) algorithm to data acquired from a LiDAR sensor and a vision sensor.

[0035] The tracked agricultural transport robot according to the present invention also has another technical feature in that it improves the efficiency of repetitive driving path recognition and autonomous driving by adopting a driving and steering method that follows a set path derived through driving path learning based on a SLAM algorithm.

[0036] FIG. 1 is a block mixed perspective view for specifically explaining the structure and components of an endless track type agricultural transport robot according to the present invention.

[0037] As illustrated in FIG. 1, the tracked agricultural transport robot according to the present invention includes, as essential components: a loading unit (100); a seesaw lifting unit (200); a tracked driving unit (300); a sensor unit (400); a worker identification recognition data generation unit (500); a driving path learning unit (600); a memory unit (700); a communication unit (800); and a control unit (900). It operates in any one of the following modes: a worker-following mode in which it drives and steers by following the movement of a worker; a path-following mode in which it drives and steers by following a set path obtained through driving path learning; and a manual control mode in which it drives and steers under the manual control of a worker by a remote controller.

[0038] The loading section (100) is a frame structure that serves to transport crops harvested according to agricultural work or agricultural materials used for harvesting crops, and includes a predetermined loading space formed by a guide designed to be positioned vertically in the height direction to prevent the loading from falling out.

[0039] The guide of the loading section (100) may have a height within the range of, for example, 15 to 45 cm or 15 to 35 cm.

[0040] Below the loading section (100), a seesaw lifting section (200) designed to lift the loading section (100) upward is located.

[0041] Specifically, the seesaw lifting unit (200) is designed to be located at the bottom of the loading unit (100) to move the height of the loading unit (100) upward.

[0042] Specifically, the seesaw lifting unit (200) may include a scissor-type lift located at the bottom of the loading unit (100) and at the top of the crawler-type drive unit (300), and a hydraulic cylinder that provides pressure to raise or lower the scissor-type lift.

[0043] The scissor lift included in the seesaw lifting part (200) may have a single-stage or multi-stage structure.

[0044] In one example, the scissor lift included in the seesaw lifting part (200) may have a 1 to 3-stage structure.

[0045] In a specific example, the seesaw lifting unit (200) may include a scissor-type lift consisting of 1 to 3 stages; and a hydraulic cylinder that provides pressure to raise or lower the scissor-type lift. The hydraulic cylinder may be able to operate by receiving an electrical signal converted into a quantitative hydraulic control signal.

[0046] The operation of the seesaw lifting unit (200) can be implemented by applying pressure to a hydraulic cylinder through upward lifting control of the loading box (100) of the control unit (900) via a signal provided from a remote controller, for example, so that the scissor lift rises or falls.

[0047] At the bottom of the seesaw lifting unit (200), an endless track type drive unit (300) is located, which receives power from a drive motor and provides forward and backward movement and left and right steering force for the robot.

[0048] As shown in FIG. 4, the track-type drive unit (300) includes a plurality of track rollers (301) that receive power from a drive motor and drive, a track shoe (302) that rotates on the track by the plurality of track rollers (301), and a roller frame (303) that fixes the track rollers.

[0049] A plurality of track rollers (301) included in the crawler-type drive unit (300) may consist of, for example, 5 to 10 lower track rollers, 5 to 10 track rollers, and 2 or more connecting rollers, and each position or arrangement may be freely designed and modified by a person with ordinary knowledge of this technical field.

[0050] The track shoe (302) included in the endless track type drive unit (300) may be a rubber or synthetic resin chain-type belt having a width corresponding to the specifications of the plurality of track rollers (301).

[0051] The roller frame (303) included in the endless track type drive unit (300) is configured to fix a plurality of track rollers (301), and any known roller frame structure designed to fix and roll a plurality of track rollers (301) can be used without limitation.

[0052] The track-type drive unit (300) can be driven forward and backward and driven left and right by the control unit (900), and accordingly, a plurality of track rollers (301) and track shoes (302) included in the track-type drive unit (300) can be driven.

[0053] The drive motor that provides power to the crawler-type drive unit (300) is driven by receiving electric energy from a battery that provides electric energy.

[0054] The capacity of the battery providing electric power to the crawler-type drive unit (300) is not particularly limited, and it is preferable that it be designed so that the usage time after full charge is within the range of 8 to 12 hours, for example, within the range of 6 to 10 hours. The type of battery is also not particularly limited, and for example, an AMG battery, LEP battery, NCM battery, or a combination thereof may be used.

[0055] The tracked agricultural transport robot according to the present invention performs driving and steering that follows a worker or follows a set path based on a sensor unit (400) including a lidar.

[0056] Specifically, the sensor unit (400) includes a LiDAR sensor (401) that acquires surrounding shape information and the relative position of surrounding objects in the form of 3D point cloud data according to driving and steering, as shown in FIG. 5, a vision sensor (402) that acquires RGB-D data regarding surrounding shape information and surrounding objects according to driving and steering, a GPS sensor (403) that acquires the real-time position of the robot according to driving and steering, and an inertial measurement device (404) that acquires tilt data in the yaw, roll, and pitch directions in real time.

[0057] A lidar sensor (401) is a device that measures surrounding shape information and the relative position of surrounding objects by irradiating near-infrared light, visible light, or ultraviolet light to the surroundings and detecting the reflected light through a light sensor. The lidar sensor (401) used in the present invention obtains surrounding shape information and the relative position of surrounding objects in the form of three-dimensional point cloud data. The three-dimensional point cloud data obtained from the lidar sensor (401) is used to generate worker classification recognition data in the worker classification recognition data generation unit (500), and is also utilized as learning data in the driving path learning unit (600).

[0058] The number of channels of the lidar sensor (401) is not particularly limited as long as it is sufficient for the acquired 3D point cloud data to be used as data in the worker classification recognition data generation unit (500) and the driving path learning unit (600), and for example, 1, 4, 16, 32, or 64 channels of lidar sensors may be used.

[0059] The vision sensor (402) is configured to acquire surrounding shape information and RGB-D data of surrounding objects according to driving and steering, and can be used to generate data that can be calibrated to the same coordinate system as the 3D point cloud data acquired from the lidar sensor (401).

[0060] In one example, the vision sensor (402) may be an RGB-D camera designed to acquire RGB color data and object distance data.

[0061] RGB-D data obtained from the vision sensor (402) is used to generate worker identification recognition data in the worker identification recognition data generation unit (500).

[0062] The GPS sensor (403) can be any sensor that acquires real-time position data of the robot based on driving and steering. In one example, the GPS sensor (403) may be a sensor capable of acquiring RTK GPS data to determine the real-time position of the robot based on driving and steering.

[0063] Specifically, the GPS sensor (403) may be an RTK-GNSS GPS sensor designed to receive GPS correction data via a communication frequency such as LTE and acquire RTK GPS data.

[0064] Data obtained from the GPS sensor (403) can be used to correct the accuracy of a path map including a set path performed by the driving path learning unit (600), and can also be used for driving that follows a set path when implementing an autonomous driving mode performed under the control of the control unit (900).

[0065] The inertial measurement device (404) may be configured to acquire tilt data in the yaw, roll, and pitch directions in real time and may be used as a so-called IMU sensor.

[0066] Specifically, the inertial measurement unit (404) may be a 9-axis IMU sensor consisting of an acceleration sensor, an angular velocity sensor, and a geomagnetic sensor. The tilt data in the yaw, roll, and pitch directions obtained from the inertial measurement unit (404) may also be used to correct the accuracy of a path map including a set path performed by the drive path learning unit (600).

[0067] The data obtained from the sensor unit (400) is used to generate worker classification recognition data in the worker classification recognition data generation unit (500) or is utilized as learning data in the driving path learning unit (600).

[0068] The worker identification recognition data generation unit (500) generates worker identification recognition data through a Multiple Object Tracking (MOT) algorithm using 3D point cloud data acquired by a LiDAR sensor (401) and RGB-D data acquired by a vision sensor (402).

[0069] The Multiple Object Tracking (MOT) algorithm performed in the worker classification recognition data generation unit (500) can generate worker classification recognition data through the Multiple Object Tracking (MOT) algorithm for the acquired data to be analyzed by calibrating the data individually acquired from the LiDAR sensor (401) and the vision sensor (402).

[0070] Specifically, as illustrated in FIG. 6, the worker classification recognition data generation unit (500) can generate worker classification recognition data by individually extracting feature points from each of the 3D point cloud data and RGB-D data, performing coordinate system transformation, and then applying a Multiple Object Tracking (MOT) algorithm to the data obtained through calibration using a transformation matrix and an inner / outer matrix.

[0071] The Multiple Object Tracking (MOT) algorithm performed in the worker classification recognition data generation unit (500) may utilize, for example, Deep-SORT, AB3DMOT, UCMCTrack, and Joint Detection and Embedding.

[0072] In a specific example, the worker classification recognition data generation unit (500) can generate worker classification recognition data by individually extracting feature points from each of the 3D point cloud data and RGB-D data, performing coordinate system transformation, and then applying the data obtained through calibration using a transformation matrix and an inner / outer matrix to one of the multiple object tracking (MOT) algorithms selected from Deep-SORT, AB3DMOT, UCMCTrack, and Joint Detection and Embedding.

[0073] In a more specific example, the worker classification recognition data generation unit (500) can generate worker classification recognition data by applying a Multiple Object Tracking (MOT) algorithm according to a detection-based tracking technique, which extracts feature points individually from each of the 3D point cloud data and RGB-D data, performs coordinate system transformation, performs object detection by applying a YOLO model to the data obtained through calibration using a transformation matrix and an inner / outer matrix, extracts features by applying a convolutional neural network model, and performs object tracking by applying a Kalman filter and a Hungarian algorithm.

[0074] The worker classification recognition data generation unit (500) can also perform a deep learning algorithm based on classifying human figures as workers, but selecting and recognizing a target worker to be followed when multiple human figures exist.

[0075] In one example, the worker classification recognition data generation unit (500) can generate worker classification recognition data by applying a Multiple Object Tracking (MOT) algorithm having a feature extraction process by a convolutional neural network model that uses data about the target worker as training data to the data obtained through calibration using a transformation matrix and an inner / outer matrix, after individually extracting feature points from each of the 3D point cloud data and RGB-D data and performing coordinate system transformation.

[0076] The worker classification recognition data generated by the worker classification recognition data generation unit (500) may include data regarding the worker to be followed, and based on the data regarding the worker to be followed, the control unit (900) may control the infinite track type drive unit (200) for worker tracking.

[0077] The three-dimensional point cloud data obtained by the LiDAR sensor (401) is also used as driving path learning data for the driving path learning unit (600).

[0078] Specifically, the driving path learning unit (600) applies a Simultaneous Localization And Map-Building (SLAM) algorithm to three-dimensional point cloud data obtained by the LiDAR sensor (401) to generate a path map including a set path and performs driving path learning to estimate the driving path of the robot within the path map.

[0079] The SLAM (Simultaneous Localization And Map-Building) algorithm used for driving path learning performed in the driving path learning unit (600) is an algorithm for simultaneous location tracking and map building, and as shown in FIG. 7, it may consist of a front-end unit that tracks the real-time location of a robot using 3D point cloud data obtained from a LiDAR sensor (401) and a back-end unit that performs map building.

[0080] The SLAM (Simultaneous Localization And Map-Building) algorithm used for driving path learning performed in the driving path learning unit (600) may be a known SLAM algorithm such as LOAM, LeGO-LOAM, HDL GRAPH SLAM, or LIO-SAM.

[0081] In one example, the driving path learning unit (600) can perform driving path learning by applying a Simultaneous Localization And Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to three-dimensional point cloud data acquired by a LiDAR sensor (401) to generate a path map including a set path and estimating the driving path of the robot within the path map.

[0082] The driving path learning unit (600) may also use data obtained from a GPS sensor (403) or an inertial measurement unit (404) to generate a path map with improved accuracy.

[0083] In one specific example, the driving path learning unit (600) can apply a Simultaneous Localization And Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to 3D point cloud data obtained by a LiDAR sensor (401), and apply a Simultaneous Localization And Map-Building (SLAM) algorithm using location data obtained by a GPS sensor (403) to generate a path map including a set path for which accuracy correction is performed, and can perform driving path learning to estimate the driving path of a robot within the path map.

[0084] In another example, the driving path learning unit (600) can apply a Simultaneous Localization and Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to three-dimensional point cloud data acquired by a LiDAR sensor (401), and generate a path map including a set path in which accuracy correction is performed by applying the SLAM algorithm using tilt data in the yaw, roll, and pitch directions acquired by an inertial measurement device (404), and perform driving path learning to estimate the driving path of a robot within the path map.

[0085] In another example, the driving path learning unit (600) can apply a Simultaneous Localization and Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to three-dimensional point cloud data obtained by a LiDAR sensor (401), and can generate a path map including a set path with accuracy correction performed by applying the SLAM algorithm using position data obtained by a GPS sensor (403) and tilt data of the yaw, roll, and pitch directions obtained by an inertial measurement unit (404), and can perform driving path learning to estimate the driving path of a robot within the path map.

[0086] As described above, rather than applying the SLAM (Simultaneous Localization And Map-Building) algorithm using only the 3D point cloud data obtained by the LiDAR sensor (401), it is preferable to apply the SLAM (Simultaneous Localization And Map-Building) algorithm using one or more of the position data obtained by the GPS sensor (403) and the tilt data of the yaw, roll, and pitch directions obtained by the inertial measurement device (404) in terms of the accuracy of generating a path map including a set path.

[0087] Through the driving path learning of the driving path learning unit (600) as described above, the open-field path that was driven by following the operator or by the operator manually controlling it can be recognized as a repeated driving path, and this can be mapped into a path map including a set path so that it is recognized as a target path to follow when implementing the autonomous driving mode. Meanwhile, the path map including the set path generated according to the driving path learning result of the driving path learning unit (600) may be composed of a single or multiple path maps, and any one of them may be selected as a target path to follow according to the operator's set signal provided by the communication unit (700).

[0088] The worker classification recognition data generated by the worker classification recognition data generation unit (500) and the data based on the learning result of the driving path learning unit (600) are stored in the memory unit (700) and retrieved under the control of the control unit (900) and can be used as data for controlling the endless track type driving unit (200).

[0089] Specifically, the memory unit (700) stores a path map including worker classification recognition data generated by the worker classification recognition data generation unit (500) and a setting path generated by the driving path learning unit (600).

[0090] The worker classification recognition data generated by the worker classification recognition data generation unit (500) may include data regarding the target worker to be followed, and the path map including the set path generated by the driving path learning unit (600) may include multiple path maps, and such data may also be stored in the memory unit (700).

[0091] In one example, the memory unit (700) can store worker classification recognition data generated by the worker classification recognition data generation unit (500) and including data about the target worker to be followed, and can store a plurality of path maps generated by the driving path learning unit (600).

[0092] The crawler-type agricultural transport robot according to the present invention receives a work mode setting signal or driving and steering signals based on manual control from a remote controller operated by the operator. The setting signal provided by the remote controller is received via a communication unit (800) and can serve as a control signal for the control unit (900).

[0093] Specifically, the communication unit (800) receives a setting signal for any one of the following modes: a worker-following mode, an autonomous driving mode, and a manual control mode, as a setting signal provided by the worker to the remote controller.

[0094] The communication unit (800) can also receive additional setting signals required for driving and steering of the robot along with a setting signal for the work mode according to the work mode.

[0095] In one example, the communication unit (800) may receive any one of the following signals: a setting signal for a worker following mode including a worker to be followed and a following distance range for said worker to be followed, which is included in the worker identification recognition data stored in the memory unit (700) as a setting signal provided by the worker to the remote controller; a setting signal for an autonomous driving mode including any one of a plurality of path maps stored in the memory unit (700) as a setting signal provided by the worker to the remote controller; and a setting signal for a manual control mode including data for driving and steering directions. In this case, the control unit (900) controls the driving of the track-type drive unit (200) so that the target worker to be followed can be followed within the following distance range when the setting signal received from the communication unit (800) is in the worker following mode, controls the track-type drive unit (200) so that it follows a set path within one of the multiple path maps stored in the memory unit (700) when the setting signal received from the communication unit (800) is in the autonomous driving mode, and controls the track-type drive unit (200) according to the data regarding the driving and steering directions when the setting signal received from the communication unit (800) is in the manual control mode.

[0096] The communication unit (800) can also transmit data acquired from the sensor unit (400), worker classification recognition data generated from the worker classification recognition data generation unit (500), learning results from the driving path learning unit (600), data stored in the memory unit (700), and results regarding the control status of the control unit (900) to the remote controller.

[0097] The method by which the communication unit (800) transmits and receives setting signals or data from the remote controller is not particularly limited, and, for example, can transmit and receive setting signals or data through a Bluetooth module, Wifi, Zigbee communication module, 4G or 5G communication module.

[0098] Meanwhile, the tracked agricultural transport robot according to the present invention may be equipped with a display that allows a worker to visually check data acquired from the sensor unit (400), worker classification recognition data generated from the worker classification recognition data generation unit (500), learning results from the driving path learning unit (600), data stored in the memory unit (700), and results related to the control status of the control unit (900).

[0099] In one example, the above-described tracked agricultural transport robot may further include a display unit designed to allow a worker to visually check data acquired from the sensor unit (400), worker identification recognition data generated from the worker identification recognition data generation unit (500), learning results from the driving path learning unit (600), data stored in the memory unit (700), and results related to the control status of the control unit (900). The display unit may be located on the front, side, or rear of the robot in the form of a monitor, and the worker can check the operation and control status of the robot through the display unit during or before and after work.

[0100] The tracked agricultural transport robot according to the present invention can be operated and driven under the control of a control unit (900).

[0101] Specifically, the control unit (900) controls the tracked drive unit (200) to follow the worker based on the worker identification recognition data stored in the memory unit (700) when the setting signal received from the communication unit (800) is in the worker following mode, controls the tracked drive unit (200) to follow the set path within the path map stored in the memory unit (700) when the setting signal received from the communication unit (800) is in the autonomous driving mode, and controls the tracked drive unit (200) to drive and steer according to the drive control signal provided by the worker through the remote controller when the setting signal received from the communication unit (800) is in the manual control mode.

[0102] In one specific example, the control unit (900) can control the infinite track type drive unit (200) to distinguish and follow a worker within 3D cloud point data obtained in real time from the LiDAR sensor (401) included in the sensor unit (400) based on the worker distinction recognition data stored in the memory unit (700) when the setting signal received from the communication unit (800) is in worker following mode.

[0103] In another example, the control unit (900) can control the infinite track type drive unit (200) to distinguish and follow a worker in RGB-D data acquired in real time from a vision sensor (402) included in the sensor unit (400) based on the worker distinction recognition data stored in the memory unit (700) when the setting signal received from the communication unit (800) is in worker tracking mode.

[0104] In another specific example, when the setting signal received from the communication unit (800) is in autonomous driving mode, the control unit (900) can control the infinite track type drive unit (200) by comparing the position of the robot acquired in real time by the GPS sensor (403) with the position data of the set path within the path map stored in the memory unit (700) so that the real-time position of the robot follows the position data of the set path within the path map.

[0105] The control unit (900) can also control the lifting operation of the seesaw lifting unit (200) so that the loading box (1000) is lifted upward or downward.

[0106] In a specific example, the control unit (900) can control the seesaw lifting unit (200) according to an upward lifting control signal of the loading box (100) provided from a remote controller.

[0107] In a more specific example, the control unit (900) can control the amount of hydraulic pressure provided to the hydraulic cylinder included in the seesaw lifting unit (200) according to the upward lifting control signal of the loading box (100) provided from the remote controller.

[0108] Furthermore, the control unit (900) can control each component not only to control the operation of the endless track type drive unit (200) as described above, but also to transmit data, learning results, and information obtained from the components of the sensor unit (400), the worker classification recognition data generation unit (500), the drive path learning unit (600), and the memory unit (700) to the communication unit (800).

[0109] The present invention also relates to a control system for a transport robot for open-field agriculture. The control system for a transport robot for open-field agriculture comprises: a tracked agricultural transport robot (1); and a remote controller (2) that transmits a control signal to cause the tracked agricultural transport robot (1) to operate in any one of a worker-following mode, a path-following mode, and a manual control mode. The method of mutually transmitting and receiving setting signals or data between the remote controller (2) and the tracked agricultural transport robot (1) may be carried out via a communication unit (800) of the tracked agricultural transport robot (1), and the method of transmitting and receiving setting signals or data by the communication unit (800) is as described above.

[0110] The remote controller (2) may include an interface for a worker to set a work mode, an interface for operating driving and steering in manual control mode, an interface for monitoring the control status of the control unit (900), and an interface for visually checking data received from the crawler-type agricultural transport robot (1).

[0111] Although the tracked agricultural transport robot and the tracked agricultural transport robot control system including the same according to the present invention have been described in detail above with reference to the drawings, this is merely an example according to the present invention, and it is obvious to those skilled in the art that the above examples do not limit the scope of the rights of the present invention. Explanation of the symbols

[0112] 1: Tracked agricultural transport robot 100 : Loading section 200 : Seesaw lifting section 300: Tracked drive unit 301: Multiple track rollers 302 : Track Shoe 303: Roller Frame 400 : Sensor section 401: LiDAR sensor 402: Vision Sensor 403 : GPS sensor 404: Inertial Measurement Unit 500: Worker Classification Recognition Data Generation Unit 600 : Driving path learning unit 700 : Memory section 800 : Communications Department 900 : Control unit

Claims

Claim 1 A tracked agricultural transport robot operating in any one of the following modes: a worker-following mode for driving and steering in accordance with the movements of a worker, an autonomous driving mode for driving and steering in accordance with a preset path, and a manual control mode for driving and steering under the control of a worker via a remote controller; a loading section including a predetermined loading space formed by a guide designed to be positioned vertically in the height direction to prevent the cargo from falling out; a seesaw lifting section located below the loading section and designed to move the height of the loading section upward; a tracked drive section including a plurality of track rollers driven by receiving power from a drive motor, a track shoe that rotates on an orbit by the plurality of track rollers, and a roller frame that fixes the track rollers; and a sensor section including a LiDAR sensor that acquires surrounding shape information and 3D point cloud data of surrounding objects in accordance with driving and steering, a vision sensor that acquires surrounding shape information and RGB-D data of surrounding objects in accordance with driving and steering, a GPS sensor that acquires real-time position in accordance with driving and steering, and an inertial measurement device that acquires tilt data in the yaw, roll, and pitch directions in real time. A worker classification recognition data generation unit that generates worker classification recognition data through a Multiple Object Tracking (MOT) algorithm using 3D point cloud data acquired by the LiDAR sensor and RGB-D data acquired by the vision sensor; and a driving path learning unit that performs driving path learning by applying a Simultaneous Localization And Map-Building (SLAM) algorithm to 3D point cloud data acquired by the LiDAR sensor to generate a path map including a set path and estimating the driving path of a robot within the path map.A tracked agricultural transport robot comprising: a memory unit storing a path map including worker classification recognition data generated by the worker classification recognition data generation unit and a set path generated by the drive path learning unit; a communication unit receiving a setting signal for any one of a worker following mode, an autonomous driving mode, and a manual control mode as a setting signal provided by a worker via the remote controller; and a control unit controlling the tracked drive unit to follow a worker based on the worker classification recognition data stored in the memory unit when the setting signal received by the communication unit is the worker following mode, controlling the tracked drive unit to follow a set path within the path map stored in the memory unit when the setting signal received by the communication unit is the autonomous driving mode, and controlling the tracked drive unit to drive and steer according to a drive control signal provided by the worker via the remote controller when the setting signal received by the communication unit is the manual control mode. Claim 2 In claim 1, the seesaw lifting unit comprises a scissor-type lift consisting of 1 to 3 stages; and a hydraulic cylinder providing pressure to raise or lower the scissor-type lift, for an endless track agricultural transport robot. Claim 3 In claim 1, the worker classification recognition data generation unit generates worker classification recognition data by individually extracting feature points from each of the 3D point cloud data and RGB-D data, performing coordinate system transformation, and then applying the data obtained through calibration using a transformation matrix and an inner / outer matrix to any one of the Multiple Object Tracking (MOT) algorithms selected from Deep-SORT, AB3DMOT, UCMCTrack, and Joint Detection and Embedding. Claim 4 In claim 1, the worker classification recognition data generation unit generates worker classification recognition data by applying a Multiple Object Tracking (MOT) algorithm according to a detection-based tracking technique, which performs object detection by applying a YOLO model to data obtained through calibration using a transformation matrix and an inner / outer matrix after individually extracting feature points from each of the 3D point cloud data and RGB-D data and performing coordinate system transformation, and extracts features by applying a convolutional neural network model, and performs object tracking by applying a Kalman filter and a Hungarian algorithm. Claim 5 A tracked agricultural transport robot according to claim 1, wherein the driving path learning unit applies a Simultaneous Localization and Map-Building (SLAM) algorithm selected from LOAM, LeGO-LOAM, HDL GRAPH SLAM, and LIO-SAM to 3D point cloud data acquired by the LiDAR sensor, and generates a path map including a set path for which accuracy correction is performed by applying the SLAM algorithm using tilt data in the yaw, roll, and pitch directions acquired by the inertial measurement device, and performs driving path learning to estimate the driving path of the robot within the path map.

Images