Farm robot's rail boarding control device
The rail boarding control device for farmbots uses sensors to set an entry line and point, addressing the challenges of GPS accuracy and high-cost LiDAR, enabling precise rail boarding in smart farms.
Patent Information
- Application Number
- KR1020200152128
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-11-13
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2040-11-13
AI Technical Summary
Existing farmbots face challenges in accurately boarding rails in smart farms due to reduced GPS accuracy indoors and the high cost and environmental constraints of using high-performance LiDAR, with methods like burying magnets or attaching QR codes being difficult in non-standardized agricultural environments.
A rail boarding control device for farmbots using sensors like cameras, mid-to-low-cost LiDAR, and depth cameras to set an entry line and point, utilizing a sensor unit, detection unit, control unit, and driving unit to ensure precise rail boarding.
Enables accurate and cost-effective rail boarding for farmbots by setting an entry line and point using commonly available sensors, enhancing their ability to ride on rails in smart farms.
Smart Images

Figure 112020121917175-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a device and method for detecting and determining control for a farm bot to ride on a rail, particularly when a rail is installed on a driving path, in the use of a farm bot (farm robot) in a farm (smart farm). Background Technology
[0002] The adoption of smart farms is becoming more widespread, and the use of farmbots is increasing.
[0003] Some greenhouse-type smart farms install rails on the floor to assist in the movement of transport carts.
[0004] In operating farm bots in such smart farms, for the bots to move autonomously, not only general autonomous driving technology but also technology for boarding rails as needed is required.
[0005] However, while precise location setting is required for the FarmBot to board the rail, and accurate positioning is possible outdoors using GPS-RTK, there is a problem where accuracy drops significantly indoors.
[0006] Alternatively, while mapping and localization can be performed using high-performance LiDAR, this is costly and often subject to environmental constraints that prevent the use of such high-performance LiDAR.
[0007] Alternatively, while it is common to utilize methods such as burying magnets or attaching marks like QR codes to the ground or equipment, there are difficulties in applying these methods in non-standardized agricultural environments. The problem to be solved
[0008] The present invention aims to solve the problems of the past by using commonly used sensors, cameras, mid-to-low-cost LiDAR, depth cameras, etc., to set an entry line and an entry point (position, direction) for the farm bot to ride on a rail, and to provide a control device and method for the farm bot to move to the entry point.
[0009] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem
[0010] A rail boarding control device for a farmbot according to an embodiment of the present invention for achieving the above objective comprises: a sensor unit that acquires data frames at regular time intervals for recognizing surrounding conditions; a detection unit that detects a rail using the data frames acquired by the sensor unit and sets line information of the rail; a control unit that sets an entry line using the line information of the rail, converts the entry line into a real coordinate system, sets an entry point on the entry line, and sets driving information for moving to the entry point; and a driving unit that moves the farmbot according to the driving information set by the control unit.
[0011] The above control unit can set an entry line using one or more pieces of information, such as rail line information, installation location information of the sensor unit stored in the storage unit, location information of the farmbot, characteristic information of the sensor, and rail information.
[0012] And, it is preferable that the sensor unit be one of a camera sensor, a LiDAR sensor, or a DEPTH camera. Effects of the invention
[0013] According to one embodiment of the present invention, by utilizing an image information acquisition sensor to set an accurate entry line and entry point for a farm bot to ride on a rail in a farm or smart farm where a rail is installed, and by controlling a drive unit, the farm bot can ride on the rail. Brief explanation of the drawing
[0014] FIG. 1 is a block diagram illustrating a rail boarding control device for a farm bot according to the present invention. FIGS. 2A and 2B are reference diagrams for explaining an embodiment of control for rail riding of the PalmBot of the present invention. FIGS. 3 to 5 are reference diagrams for explaining the process of detecting a rail in an image frame in one embodiment of the present invention. FIG. 6 is a reference diagram for explaining an example of setting line information (y=ax+b) of a rail detected in one embodiment of the present invention. FIG. 7 is a flowchart for explaining a rail boarding control method of a farmbot according to one embodiment of the present invention. Specific details for implementing the invention
[0015] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components, steps, actions, and / or elements in addition to the mentioned components, steps, actions, and / or elements.
[0016] FIG. 1 is a block diagram illustrating a rail boarding control device for a farm bot according to the present invention.
[0017] A rail boarding control device for a farm bot according to one embodiment of the present invention includes a sensor unit (100), a detection unit (200), a control unit (300), and a driving unit (400).
[0018] The sensor unit (100) acquires data frames at regular time intervals to determine the surrounding environment.
[0019] The detection unit (200) detects a rail in a data frame obtained by the sensor unit (100) and sets line information in the detected rail.
[0020] The control unit (300) sets an entry line using the line information of the rail and converts the entry line into a real coordinate system. To this end, a MAP implemented through a real coordinate system (world coordinate system or robot coordinate system) is stored and managed in the storage unit (500).
[0021] And the control unit (300) converts the entry line detected by the coordinate system (image coordinate system) of the sensor unit (100) into an entry line matched to the real coordinate system based on the real coordinate system of the MAP stored in the storage unit (500).
[0022] In addition, the control unit (300) uses the rail line information to set an entry point on the entry line based on the actual entry line matched to the converted real coordinate system for the farm bot to ride the rail and the farm bot's current GPS position (P), and sets driving information for moving to the entry point.
[0023] The driving unit (400) moves the farm bot based on the driving information of the control unit (300).
[0025] Meanwhile, a storage unit (500) according to one embodiment of the present invention stores a MAP created based on a real coordinate system and includes information on the installation location and installation direction of the sensor unit (100) relative to the reference point (center point) of the farm bot.
[0026] And the storage unit (500) may include rail information such as the thickness of the rail, rail width, and height relative to the ground, and may include sensor characteristic information such as the characteristics of the sensor unit (100), that is, in the case of a camera, resolution, focal length, FOV, pixel size, pixel size, etc.
[0027] And, the storage unit (500) may include shape information such as the size of the farm bot.
[0029] FIGS. 2a and 2b are reference diagrams for explaining an embodiment of control for rail riding of the PalmBot of the present invention.
[0030] In this embodiment, the sensor unit (100) is assumed to be a camera, and an embodiment is described in which a palm bot (201) moves to the vicinity of a boarding rail by autonomous driving or other means to board the rail.
[0031] First, the camera, which is the sensor unit (100), captures an image frame containing a rail to be ridden (208, FIG. 3) and provides it to the detection unit (200).
[0032] Then, the detection unit (200) can perform the detection process by setting a certain area (a) as shown in FIG. 4 as needed.
[0033] Afterward, the detection unit (200) detects the rail (202) in the captured image frame as shown in FIG. 5. Here, the rail detection technology can be easily implemented by utilizing a lane detection technology for preventing lane departure or a lane keeping device of an automobile.
[0034] The detection unit (200) sets the line information of the detected rail (e.g., y=ax+b) as shown in FIG. 6.
[0035] The detection unit (200) provides one or more line information, such as left line information (203: yl=a1*x+b1) or right line information (204: yr=a2*x+b2) of the rail.
[0036] The line information of the rail detected by the detection unit (200) can be set after detecting the edge of the rail.
[0037] The control unit (300) sets the entry line (205) using one or more line information on the left or right side of the rail.
[0038] Meanwhile, when the camera, which is a sensor unit (100) installed on the farm bot (201), is positioned in the center, the entry line can be set using the method of [Mathematical Formula 1] below.
[0039] [Mathematical Formula 1]
[0040] Entry line = -(b1 / a1+b2 / a2)(2)x +(b1+b2)(2), if a1 and a2 have the same sign,
[0042] In contrast, if the camera is not located in the exact center of the palm bot (201), a corrected entry line is set by taking into account the installation location of the sensor.
[0043] The control unit (300) can set the real coordinate system entry line by taking into account the characteristics of the rail.
[0044] The control unit can set the entry line in various ways using line information.
[0046] As shown in FIG. 2a, the control unit (300) can limit the section where the farm bot (201) can enter based on its current position, the driving characteristics of the farm bot (201), and the presence or absence of obstacles at the entry line.
[0048] Accordingly, the control unit (300) can set the entry point differently depending on the movement method, entry distance, and movement efficiency.
[0049] First, if the farm bot (2010) needs to move in one dimension (x-axis) from its current position (P), the entry point {X} (206) on the entry line is set as the entry point.
[0050] Meanwhile, if the control unit (300) can move in two dimensions (x-axis, y-axis) from the current position (P), it can set the nearest entry point 2{X2, Y2}(207) on the entry line as the entry point.
[0051] On the other hand, the control unit (300) may set a 2D entry point 3{X3, Y3} (209) as the optimal entry point on the entry line (205) by considering the current position of the farm bot (201), the driving characteristics and movement efficiency of the farm bot (201).
[0053] Typically, there are various methods such as the RRT (Rapid Exploring Random Tree) algorithm, which constructs a map including obstacles and sets the optimal entry point and the path to the entry point from the current location.
[0055] The control unit (300) sets driving information for moving to the entry point relative to the current position (P).
[0056] At this time, the control unit (300) can set driving information using one or more pieces of information such as the movement speed, movement direction, and movement path of the farm bot (201).
[0057] And the control unit (300) sets driving information, such as an optimal movement path, based on the current location and entry point of the farm bot (201).
[0058] The control unit (300) can simultaneously set the optimal entry point and driving information by considering the driving characteristics and movement efficiency of the farm bot (201).
[0059] The control unit (300) provides a control signal to the driving unit (400) to drive the farm bot (201) based on driving information.
[0060] The driving unit (400) moves the farmbot (201) to the entry point according to the control signal.
[0061] The control unit (300) can set the entry line using rail shape information such as rail width by using one line information of the left or right side of the rail.
[0063] The control unit (300) can set an entry line (205) that takes into account the installation location of the camera.
[0064] The control unit (300) can set an entry line (205) that takes into account the current position of the farm bot (201).
[0065] The control unit (300) can set the entry line (205) using characteristic information of the sensor unit (100) (e.g., in the case of a camera, the resolution of the image sensor, focal length, pixel size, FOV, etc.).
[0066] In the present invention, the term "position" includes {X, Y} or {X, Y, Z} position and direction information, as shown in FIG. 2b.
[0067] The control unit (300) can set driving information to go to the entry point in one go, and can set driving information to move to the entry point gradually as needed.
[0069] Typically, rail line information or entry line information in a video image is used by converting the information of the image sensor's coordinate system into a two-dimensional real coordinate system based on the robot.
[0070] The position, direction, and characteristics of the sensor relative to the reference point of the farm bot (201) are converted, and the information is converted, set, and utilized based on the real coordinate system (robot coordinate system or world coordinate system) of the farm bot (201).
[0071] This transformation belongs to the category of skills utilized by engineers at an average level.
[0072] Additionally, the entry point location information for the movement of the farm bot (201) can use absolute coordinates, and can be set and utilized as relative coordinates for the position of the farm bot (201).
[0073] In Fig. 7, determining whether the entry line or boarding is complete can be done when alpha + beta = 0 or a value near 0.
[0074] In addition, the above invention can move to the entry point in a single step, but can also move to the entry point through repeated execution.
[0076] Hereinafter, a rail boarding control method for a palm bot according to one embodiment of the present invention will be described with reference to FIG. 7.
[0077] First, the farmbot (201) moves to the vicinity of the boarding rail by autonomous driving or other means (S710).
[0078] Next, a video frame containing a rail to be ridden is captured by the sensor unit (100) (S720).
[0079] A rail (202) is detected in a captured image frame by the detection unit (200) (S730). Here, rail detection technology can be easily implemented by utilizing lane detection technology for preventing lane departure or lane keeping devices of automobiles.
[0080] Afterwards, line information of the detected rail (e.g., y=ax+b) is set through the detection unit (200) (S740). In this embodiment, the detection unit (200) provides one or more line information, such as left line information (203: yl=a1*x+b1) or right line information (204: yr=a2*x+b2) of the rail.
[0081] Next, the entry line (205) is set using the left and right line information of the rail through the control unit (300) (S750). In this embodiment, when the camera, which is the sensing unit (100) installed on the farm bot (201), is positioned in the center, the entry line can be set using the method of [Equation 1] below.
[0082] Subsequently, the control unit (300) sets the entry point in the real coordinate system by considering the characteristics of the rail (S760). Meanwhile, the control unit (300) can limit the section where the farm bot (201) can enter based on the current position of the farm bot (201), the driving characteristics of the farm bot (201), and the presence or absence of obstacles on the entry line. Accordingly, the control unit (300) can set the entry point differently depending on the movement method, entry distance, and movement efficiency.
[0083] First, if the farm bot (2010) needs to move in one dimension (x-axis) from its current position (P), the entry point {X} (206) on the entry line is set as the entry point.
[0084] Meanwhile, if the control unit (300) can move in two dimensions (x-axis, y-axis) from the current position (P), it can set the nearest entry point 2{X2, Y2}(207) on the entry line as the entry point.
[0085] On the other hand, the control unit (300) may set a 2D entry point 3{X3, Y3} (209) as the optimal entry point on the entry line (205) by considering the current position of the farm bot (201), the driving characteristics and movement efficiency of the farm bot (201).
[0086] Typically, there are various methods such as the RRT (Rapid Exploring Random Tree) algorithm, which constructs a map including obstacles and sets the optimal entry point and the path to the entry point from the current location.
[0088] In this way, the control unit (300) sets driving information for moving to an entry point relative to the current position (P) using various methods as described above (S770). At this time, the control unit (300) can set driving information using one or more pieces of information such as the movement speed, movement direction, and movement path of the farm bot (201). Then, the control unit (300) sets driving information such as an optimal movement path based on the current position and entry point of the farm bot (201).
[0089] Additionally, the control unit (300) can simultaneously set an optimal entry point and driving information by considering the driving characteristics and movement efficiency of the farm bot (201). Based on the driving information, the control unit (300) provides a control signal to the driving unit (400) to drive the farm bot (201).
[0090] Afterwards, the drive unit (400) moves the farmbot (201) to the entry point according to the control signal (S780).
[0091] Meanwhile, determining whether the entry line or boarding is complete can be done if alpha + beta = 0 or a value near 0.
[0092] The control unit (300) can determine boarding completion in various ways using rail line information.
[0093] Although the configuration of the present invention has been described in detail with reference to the accompanying drawings, this is merely illustrative, and it is obvious that those skilled in the art can make various modifications and changes within the scope of the technical concept of the present invention. Accordingly, the scope of protection of the present invention should not be limited to the aforementioned embodiments but should be determined by the description in the following claims.
Claims
Claim 1 A rail riding control device for a farmbot comprising: a sensor unit that acquires data frames at regular time intervals for recognizing surrounding conditions; a detection unit that detects a left, right, or both rails using the data frames acquired by the sensor unit and sets line information of the left, right, or both rails; a control unit that sets an entry line using the set line information of the left, right, or both rails, converts the entry line into a real coordinate system, sets an entry point on the entry line, and sets driving information for moving to the entry point; and a driving unit that moves the farmbot according to the driving information set by the control unit, and a storage unit that includes one or more pieces of information among a MAP implemented through a real coordinate system, rail information including the installation position of the sensor unit relative to the farmbot's reference point, installation direction information, rail thickness, rail width, and height relative to the ground, and sensor characteristic information. Claim 2 delete Claim 3 A rail boarding control device for a farm bot according to claim 1, wherein the control unit converts the entry line detected by the coordinate system of the sensor unit into an entry line matched to the actual coordinate system based on the actual coordinate system of the MAP stored in the storage unit, sets an entry point on the entry line based on the actual entry line matched to the converted actual coordinate system and the farm bot's current GPS position (P) using rail line information, and sets driving information for moving to the entry point. Claim 4 A rail boarding control device for a farm bot according to claim 1, wherein the detection unit receives an image frame containing a rail to be boarded captured by the sensor unit, sets a certain area (a) to detect the rail, sets left line information and right line information of the detected rail, and the control unit sets an entry line using one or more left or right line information of the rail. Claim 5 A rail boarding control device for a farmbot according to claim 4, wherein the control unit limits the section where entry is possible based on the farmbot's current position, driving characteristics, and the presence or absence of obstacles at the entry line. Claim 6 A rail boarding control device for a farm bot according to claim 1, wherein the control unit sets an entry point {X} on the entry line as the entry point when the farm bot needs to move in one dimension (x-axis) from its current position (P), and sets the nearest entry point 2 {X2, Y2} on the entry line as the entry point when it is possible to move in two dimensions (x-axis, y-axis) from its current position (P). Claim 7 A method for controlling a farmbot’s rail boarding, comprising: a step of moving the farmbot to near a boarding rail; a step of capturing an image frame containing the rail to be boarded by a sensor unit of the farmbot; a step of detecting the rail in the captured image frame by a detection unit of the farmbot; a step of detecting a left, right, or both rails by a detection unit of the farmbot and setting line information of the detected left, right, or both rails; a step of setting an entry line using the line information of both sides of the detected left, right, or both rails by a control unit of the farmbot; a step of setting a real coordinate system entry point by a control unit of the farmbot considering the characteristics of the rail; a step of setting driving information for moving from the farmbot’s current position to the entry point; and a step of moving the farmbot to the set real coordinate system entry point, wherein a storage unit includes one or more pieces of information among rail information such as a MAP implemented through the real coordinate system, installation position of the sensor unit relative to the farmbot’s reference point, installation direction information, rail thickness, rail width, and height relative to the ground, and sensor characteristic information. Claim 8 delete Claim 9 A rail boarding control method for a palm bot according to claim 7, wherein the step of moving the palm bot to a set real coordinate system entry point is characterized by determining that the palm bot is on the entry line or has completed boarding when its position is located on both set lines. Claim 10 delete
Citation Information
Patent Citations
Crop planting system
KR1020190080208A
Moving Robot and controlling method
KR1020190103512A
agricultural machinery and automatic driving method thereof
KR1020200116437A
