Apparatus and method for recognizing moving path

Abstract

A passage recognition device, a passage recognition method, and a robot including the passage recognition device are provided. The above passage recognition device includes: a passage search unit that, when image information of an environment to which a moving body is to be applied is input, sets passage characteristics for the image information and searches for a movable passage by applying a single model that searches for a shape corresponding to the passage characteristics to the image information; and a direction determination unit that determines the direction of movement of the moving body for the movable passage.

Description

Technology Field

[0001] The present invention relates to a device for recognizing a movement path, and more specifically, to a device and method for selecting a movable path using an RGB-D (Red, Green, Blue-Depth) camera and recognizing the curvature direction of the selected movable path. Background Technology

[0003] Recently, domestic industries have been establishing industrial systems that integrate Artificial Intelligence (AI) and Information and Communication Technology (ICT). This shift in industrial structure is also impacting the agricultural sector, leading to the development of new industrial systems within the field. In particular, alongside the Fourth Industrial Revolution, various research and development efforts for agricultural automation are underway, playing a crucial role in enhancing agricultural efficiency and productivity.

[0004] While autonomous driving technology is fundamentally applied to vehicles on the road, it is also being widely utilized in agricultural technology recently. Agricultural machinery equipped with autonomous driving technology measures the location and size of a field to establish a work path suitable for it, and performs tasks while moving along the set path. For example, an autonomous tractor independently measures the location and size of a field, sets a suitable work path, and then automatically performs tasks such as cultivation, sowing, and harvesting while moving along the established route.

[0005] Sensor technologies such as cameras, LiDAR (Light Detection and Ranging), and radar play a crucial role in the development of such autonomous agricultural machinery. Much research is being conducted on methods to utilize these sensors to navigate spaces where robots can move in narrow passages, such as those found in orchards or open fields. Cameras provide visual information, while LiDAR provides distance and depth data, enabling the robot to accurately perceive its surroundings; furthermore, radar enables stable detection, making the robot's movement safer.

[0006] For example, an autonomous tractor uses cameras and LiDAR to scan the terrain of a cultivated field and establishes an optimal work path based on the topographical information. Subsequently, it moves along the set path while monitoring the condition of the crops and performing tasks. During this process, the robot can analyze data in real time and detect environmental changes to modify the path or adjust the work method.

[0007] However, such technologies require additional work to reconfigure the traversable space and modify algorithms whenever the operating environment changes. For example, existing pathfinding algorithms utilizing LiDAR, cameras, and RADAR (radio detection and ranging) provide movement paths by distinguishing between traversable and non-traversable areas within a given environment. However, this approach creates the problem of having to label the robot's traversable area and retrain the model whenever it is applied to a new environment. Consequently, there are limitations to efficiently applying the same algorithm across diverse environments. Therefore, there is a need to develop flexible algorithms capable of responding to various environmental changes. Prior art literature

[0009] Korean Patent Publication No. 10-2225321 (March 3, 2021) The problem to be solved

[0010] The problem that the present invention aims to solve is to provide an apparatus and method that utilizes an RGB-D (Red, Green, Blue-Depth) camera to recognize a 'U'-shaped passageway in which a robot can move in a narrow passageway and to recognize the curvature direction of the 'U'-shaped passageway.

[0011] Furthermore, the problem that the present invention aims to solve is to provide a moving passage recognition device and method that searches for a movable passage using a single model that searches for the shape of a 'U'-shaped passage characteristic regardless of changes in the operating environment, and recognizes the direction of curvature based on changes in the shape of the movable passage.

[0012] In addition, the problem that the present invention aims to solve is to provide a device and method for recognizing a movement path that enables stable driving by determining the direction of movement of a robot based on the center point of the floor of the path. means of solving the problem

[0014] According to an embodiment of the present invention, a passage recognition device is provided. The passage recognition device is,

[0015] It includes: a passage search unit that, when image information of an environment to which a moving body is to be applied is input, sets passage characteristics for said image information and searches for a movable passage by applying a single model that searches for a shape corresponding to said passage characteristics to said image information; and a direction determination unit that determines the direction of movement of said moving body for said movable passage.

[0016] In a passage recognition device according to an embodiment of the present invention, the passage characteristic is a 'U'-shaped passage characteristic composed of a floor surface and a wall surface, and the passage search unit divides the image information into a plurality of regions at predetermined distances and applies the 'U'-shaped passage characteristic to each of the divided plurality of regions to search for the movable passage.

[0017] In a passage recognition device according to an embodiment of the present invention, the passage search unit applies a single model for searching the shape of the passage characteristics to the plurality of regions divided by distance to extract distance-specific 'U'-shaped passage characteristics, and connects neighboring 'U'-shaped passage characteristics among the extracted distance-specific 'U'-shaped passage characteristics to search for the movable passage.

[0018] In a passage recognition device according to an embodiment of the present invention, when a movable passage is detected by applying the 'U'-shaped passage characteristic to each of the plurality of divided regions, the direction determining unit can extract the center point of the floor surface existing within the plurality of distance-based 'U'-shaped passage characteristics applied to the plurality of divided regions, and determine the curvature direction of the movable passage by connecting adjacent center points among the extracted plurality of center points.

[0019] In a passage recognition device according to an embodiment of the present invention, a passage characteristic setting unit for setting the passage characteristic is additionally provided; and when at least one of height information from the floor, field of view information of a camera mounted on the moving body, and depth information of the image information is input as prior information to search for the movable passage for the image information, the passage characteristic setting unit can set a 'U'-shaped passage characteristic composed of a floor surface and a wall surface for the single model using the input prior information.

[0020] In a passage recognition device according to an embodiment of the present invention, when a 'U'-shaped movement passage is established by connecting adjacent 'U'-shaped passage characteristics among a plurality of 'U'-shaped passage characteristics applied according to distance, the direction determining unit extracts the center point of the bottom surface of the 'U'-shaped passage characteristic included in the 'U'-shaped movement passage, sets the curvature direction for the 'U'-shaped movement passage by connecting adjacent bottom surface center points among the extracted bottom surface center points, and determines the direction of movement based on the current position of the moving body based on the set curvature direction.

[0021] In a passage recognition device according to an embodiment of the present invention, the passage search unit further comprises a preprocessing unit that modifies each of the divided plurality of regions through at least one of blur, expand, and erode; and the passage search unit can search for a 'U'-shaped movement passage corresponding to the movable passage using the modified plurality of regions.

[0023] In addition, according to an embodiment of the present invention, a passage recognition method is provided. The passage recognition method performed by the passage recognition device comprises:

[0024] When image information of an environment to which a moving body is to be applied is input, the method includes: a step of setting the passage characteristics to apply a single model that searches for a shape corresponding to the passage characteristics of a movable space to the image information; a step of dividing the image information into multiple regions at predetermined distances; a step of searching for a 'U'-shaped movement passage by applying the single model that searches for the shape of the 'U'-shaped passage characteristics to the multiple divided regions; and a step of determining the direction of movement based on the shape change of the 'U'-shaped movement passage recognized by distance for the 'U'-shaped movement passage.

[0025] In a passage recognition method according to an embodiment of the present invention, the 'U'-shaped movement passage search step can search for a 'U'-shaped movement passage by applying the single model that searches for the shape of the 'U'-shaped passage characteristic to the divided plurality of regions to extract 'U'-shaped passage characteristics by distance, and by connecting neighboring 'U'-shaped passage characteristics among the extracted plurality of 'U'-shaped passage characteristics.

[0026] In a passage recognition method according to an embodiment of the present invention, the movement direction determination step determines curvature information by comparing the shape change of the recognized 'U'-shaped passage characteristic as the distance increases with the shape of the 'U'-shaped passage characteristic, and when the center points of the bottom surface of a plurality of extracted 'U'-shaped passage characteristics are in a straight line, the 'U'-shaped passage can be determined as a straight path.

[0027] In a passage recognition method according to an embodiment of the present invention, the movement direction determination step determines curvature information by comparing the shape change of the recognized 'U'-shaped passage characteristic as the distance from the shape of the 'U'-shaped passage characteristic increases, wherein if the center points of the floor surface of a plurality of extracted 'U'-shaped passage characteristics do not exist on a straight line, the 'U'-shaped passage may be determined as a curved path based on the center point.

[0028] In a passage recognition method according to an embodiment of the present invention, the movement direction determination step, when determined to be a curved path based on a change in the shape of the 'U'-shaped passage characteristic, divides the area into multiple regions by distance considering the movement speed of the moving object, applies the single model to the multiple divided regions to extract floor center points in each region, and when the number of floor center points extracted in the multiple divided regions by distance is less than or equal to a preset number, determines that the curvature is severe and can reduce the speed in the corresponding region to below a threshold speed.

[0029] In a passage recognition method according to an embodiment of the present invention, the movement direction determination step divides the space into multiple regions by distance based on movement speed and applies the single model, which searches for the shape of the passage characteristic of the movable space, to the multiple regions divided by distance, so that when the speed at which at least one of the left wall and the right wall included in the 'U'-shaped passage characteristic disappears is greater than or equal to a preset speed, the driving speed of the moving body in the movable passage can be reduced to or less than a threshold speed.

[0031] In addition, according to an embodiment of the present invention, a robot is provided. The robot communicates with a passage recognition device, and

[0032] Based on the change in the shape of the movable passage recognized by distance, the direction of movement for the movable passage is determined based on the current location. Effects of the invention

[0034] According to the present invention, the following effects are achieved.

[0035] According to an embodiment of the present invention, by setting a 'U'-shaped passage characteristic that can be utilized in various operating environments of a robot and searching for a movable passage using the 'U'-shaped passage characteristic, there is an effect of determining a movement path without resetting the algorithm or performing additional work according to the environment.

[0036] According to an embodiment of the present invention, by utilizing the characteristics of a 'U'-shaped passage to quickly and accurately recognize a movable passage, it has the effect of facilitating the movement of a robot in environments with complex structures, such as orchards or farmlands.

[0037] According to an embodiment of the present invention, by utilizing the characteristics of a 'U'-shaped passage, the curvature direction of the movement passage is recognized, and the movement path of the robot is controlled in real time.

[0038] According to an embodiment of the present invention, by recognizing the direction of curvature using the characteristics of a 'U'-shaped passage, the driving stability of the robot is enhanced and the ability to avoid obstacles is improved.

[0040] The effects obtainable from the various examples of the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly derived and understood by those skilled in the art based on the following detailed description. Brief explanation of the drawing

[0042] FIG. 1 is a configuration diagram for explaining a passage recognition system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the passage characteristics applied to recognize a moving passage independently of the operating environment in a passage recognition device according to an embodiment of the present invention. FIG. 3 is a drawing for explaining a passage recognition device according to an embodiment of the present invention. FIG. 4 is a drawing for explaining the control unit of a passage recognition device according to an embodiment of the present invention. FIG. 5 is a diagram showing the characteristics of a ‘U’-shaped passage applied according to distance and a method for determining the direction of curvature thereto according to an embodiment of the present invention, and specifically, a diagram for explaining the direction of curvature for a straight passage. FIG. 6 is a drawing showing the characteristics of a ‘U’-shaped passage applied according to distance and a method for determining the bending direction therein according to an embodiment of the present invention, and specifically, a drawing for explaining the bending direction of a curved passage. FIG. 7 is a flowchart for explaining a passage recognition method performed by a passage recognition device according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a method for a robot according to an embodiment of the present invention to determine a direction of movement based on its current position. FIG. 9 is a drawing showing a computing device according to an embodiment of the present invention. Specific details for implementing the invention

[0043] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0044] In this specification and drawings (hereinafter referred to as the 'this specification'), redundant descriptions of identical components are omitted.

[0045] Furthermore, when a component is described in this specification as being 'connected' or 'connected' to another component, it should be understood that it may be directly connected to or connected to the other component, or that there may be other components in between. On the other hand, when a component is described in this specification as being 'directly connected' or 'directly connected' to another component, it should be understood that there are no other components in between.

[0046] Furthermore, the terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention.

[0047] Additionally, in this specification, singular expressions may include plural expressions unless the context clearly indicates otherwise.

[0048] Furthermore, in this specification, terms such as 'comprising' or 'having' are intended merely to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0049] Additionally, in this specification, the term "and / or" includes a combination of the plurality of described items or any of the plurality of described items. In this specification, "A or B" may include "A," "B," or "both A and B."

[0050] Furthermore, detailed descriptions of known functions and configurations that may obscure the essence of the invention will be omitted in this specification. Hereinafter, a passage recognition device and a recognition method according to an embodiment of the present invention, and a passage recognition system including the same, will be described with reference to the drawings.

[0052] FIG. 1 is a configuration diagram for explaining a passage recognition system according to an embodiment of the present invention.

[0053] Referring to FIG. 1, the passage recognition system (10) uses image information to recognize a movable space for various operating environments or to recognize the direction of curvature according to changes in the shape of the movable passage. Through this, the passage recognition system (10) supports the robot (300) to drive stably.

[0054] The passage recognition system (10) includes a passage recognition device (100), a robot (300), an image capturing device (200), and a user terminal (500). The passage recognition device (100) uses image information received by the image capturing device (200) to set the passage characteristics (51) of a movable passage in a narrow space such as a narrow passage, and implements a single model (53) independent of the operating environment based on the passage characteristics (51). In this way, the passage recognition device (100) searches for a movable passage by applying the single model (53) that searches for a specific shape of the movable space to the image information, and controls the robot (300) to drive stably by recognizing the direction of curvature based on the shape change of the discovered movable passage. That is, the passage recognition device (100) transmits the movable passage to the robot (300), and the passage recognition device (100) supports the robot (300) to drive stably by controlling the robot (300) to control the movement target point based on the center point of the floor of the movement passage.

[0055] The passage recognition device (100) may be mounted on a robot (300). The passage recognition device (100) may be implemented in the form of a mobile robot or a vehicle, but is not limited thereto. Alternatively, when the passage recognition device (100) is mounted on a robot (300), the passage recognition device (100) may further include a plurality of sensors for acquiring surrounding terrain features, but is not limited thereto, and may acquire image information by an image capturing device (200).

[0056] The image capturing device (200) is equipped with a plurality of sensors for transmitting image information of the environment to be applied to the passage recognition device (100), and the plurality of sensors may include, for example, at least one RGB-D (Red, Green, Blue and Depth) camera or a LiDAR (Light Detection and Ranging) sensor.

[0057] The RGB-D camera provides depth information and color (RGB) information, helping the passage recognition device (100) to recognize the surrounding environment more precisely and generate a three-dimensional map. Specifically, the color image composed of the red, green, and blue channels of the RGB-D camera is used to identify objects. Additionally, the IR (infra-red) camera and IR projector of the RGB-D camera are used to calculate depth by projecting an infrared pattern onto the environment and analyzing how the infrared pattern is reflected from the surface of an object. The RGB-D camera can be used to measure the distance of each point based on the degree of deformation of the reflected pattern.

[0058] The image capturing device (200) generates image information by capturing an image containing a movable passage in an environment to be applied using at least one of an RGB-D camera or a LiDAR sensor. The image information thus generated can be provided to the passage recognition device (100).

[0059] When image information including depth information of an image is input, the passage recognition device (100) sets passage characteristics (51) for the image information to search for a 'U'-shaped movement passage and determines the direction of movement of the robot (300) for the movable passage. Through this, the robot (300) can autonomously avoid obstacles and move along the optimal path.

[0060] The robot (300) may directly collect information about the surrounding environment using an RGB-D camera, a LiDAR sensor, etc., and then move using the collected information, or it may move by a separate command from the outside by a passage recognition device (100) that provides the service. Meanwhile, the image capturing device (200) that captures depth information of an image and provides it as image information may be mounted on the passage recognition device (100), but is not limited thereto, and may be mounted on or attached to the robot (300).

[0061] When the robot (300) receives information related to the movement path from the path recognition device (100), it can determine the direction of movement based on the received information and move along the optimal path.

[0062] The passage recognition device (100) collects information regarding the direction of movement, information regarding the optimal path, information regarding the curvature direction of the movement passage, and information regarding obstacles, and generates monitoring information based on the collected information. The passage recognition device (100) transmits the generated monitoring information to a user terminal (500).

[0063] The user terminal (500) is a terminal used by the user and receives monitoring information from the passage recognition device (100). The user terminal (500) outputs the received monitoring information to help the user check the status of the curvature recognition of the movement passage in real time, even remotely. Alternatively, the user terminal (500) displays the location based on the curvature recognition of the movement passage on a map to support the user in intuitively recognizing the passage that has already been moved and the passage that has not yet been explored. Meanwhile, the user terminal (500) may be used to input at least one of the following information as preliminary information: height information regarding both walls (511) from the floor, field of view information that can be seen by the camera, and depth information of the image information received by the image capturing device (200). This may be used to set the 'U'-shaped passage characteristic (51) later, but is not limited thereto.

[0064] The user terminal (500) supports a user interface so that the user can intuitively perceive monitoring information. Preferably, the user terminal (500) may be a personal computing system such as a desktop, laptop, tablet PC, handheld PC, etc.

[0065] Meanwhile, the passage recognition system (10) supports communication between the passage recognition device (100), the robot (300), and the user terminal (500) by utilizing a network (400) established between them. The network (400) may be composed of a backbone network and a subscriber network. The backbone network may be composed of one or more integrated networks among an X.25 network, a Frame Relay network, an ATM network, an MPLS (Multi-Protocol Label Switching) network, and a GMPLS (Generalized Multi-Protocol Label Switching) network. The subscriber network may be FTTH (Fiber To The Home), ADSL (Asymmetric Digital Subscriber Line), cable network, Zigbee, Bluetooth, Wireless LAN (IEEE 802.11b, IEEE 802.11a, IEEE 802.11g, IEEE 802.11n), Wireless Hart (ISO / IEC 62591-1), ISA 100.11a (ISO / IEC 62734), CoAP (Constrained Application Protocol), MQTT (Message Queuing Telemetry Transport), WIBro (Wireless Broadband), WiMAX, 3G, HSDPA (High Speed ​​Downlink Packet Access), 4G, 5G, and 6G, etc. In some embodiments, the network (400) may be an internet network or a mobile communication network. Additionally, the network (400) may include any other widely known or future-developed wireless or wired communication methods.

[0066] In the drawing, the passage recognition device (100) is shown as being separate from the image capturing device (200) and the robot (300), but it is not limited to this and can be implemented as one device or two devices depending on the application environment in which the passage is to be recognized.

[0068] FIG. 2 is a diagram illustrating the characteristics of a passage applied to recognize a movement passage independently of the operating environment in a passage recognition device according to an embodiment of the present invention. Specifically, FIG. 2(a) is a diagram illustrating the characteristics of a passage applied to recognize a movement passage in a situation where trees are densely overgrown according to an embodiment of the present invention, FIG. 2(b) is a diagram illustrating the characteristics of a passage applied to recognize a movement passage in a situation where a vineyard exists according to an embodiment of the present invention, FIG. 2(c) is a diagram illustrating the characteristics of a passage applied to recognize a movement passage in a situation where ridges exist, such as a chili pepper field, according to an embodiment of the present invention, and FIG. 2(d) is a diagram illustrating the characteristics of a 'U'-shaped passage composed of a floor surface and a wall surface that are applied independently of the operating environment in a passage recognition device according to an embodiment of the present invention.

[0069] Referring to FIG. 2, when image information of an environment to be applied by an image capturing device (200) according to an embodiment of the present invention is input, the passage recognition device (100) sets passage characteristics (51) for the input image information, searches for a movable passage by applying a single model (53) that searches for a specific shape of a movable space to the image information, and determines the direction of movement for the movable passage. Alternatively, the passage recognition device (100) may receive image information in real time by having part or all of the image capturing device (200) mounted on the passage recognition device (100).

[0070] Here, the passageway characteristic (51) can be defined as a ‘U’-shaped passageway characteristic (51) composed of a floor surface (515) and a wall surface (511) so that it can be applied independently to the operating environment for the input video information. Alternatively, the ‘U’-shaped passageway characteristic (51) can be set by inputting prior information through a user terminal (500) before the video information is input. Alternatively, the ‘U’-shaped passageway characteristic (51) can be input by the input unit (130) of the passageway recognition device (100). The ‘U’-shaped passageway characteristic (51) input through a plurality of methods may also be set by the control unit (150) of the passageway recognition device (100).

[0071] The passage recognition device (100) can set the passage characteristics (51) of a movable passage in a narrow space, such as a narrow passage, by using either image information obtained by an image capturing device (200) or real-time image information obtained by an RGB-D camera mounted on the passage recognition device (100).

[0072] In detail, referring to FIG. 2(d), the passageway characteristic (51) can be implemented in a 'U' shape including a floor surface (515) and a wall surface (511). The 'U'-shaped passageway characteristic (51) implemented in a 'U' shape can be set using information on the maximum field of view of a camera mounted on the passageway recognition device (100) based on height information from the floor in the application environment and movement speed. The 'U'-shaped passageway characteristic (51) can be set based on the same distance from the camera of the passageway recognition device (100) to the floor surface (515), the same distance from the camera to the left wall surface (511), and the same distance from the camera to the right wall surface (511). By setting the distances from the camera to the floor surface (515), left and right walls (511) to be the same, the 'U'-shaped passageway characteristic (51) is implemented so that the passageway recognition device (100) can clearly recognize the movable passageway and subsequently determine the direction of curvature.

[0073] Additionally, the side walls (515) of the ‘U’-shaped passageway characteristic (51) are side obstacles that restrict the movement path of the moving body (300) thereafter, and the width of the passageway can be set using distance information between the side obstacles.

[0074] For example, in an orchard environment where trees are densely overgrown, the passage recognition device (100) can perform preliminary work to set 'U'-shaped passage characteristics (51), and can input 1m from the ground to the height of the tree as height information so that the narrow passage between the trees can be recognized as a passage for movement, and can input 5m, which is the maximum field of view that the camera can recognize forward, as distance information. The input height information and distance information can be used as preliminary information for setting the 'U'-shaped passage characteristics (51). In a vineyard environment, the passage recognition device (100) performs preliminary work to recognize the narrow passage between the grapevines, and in this case, 0.5m from the ground to the height of the grapevine can be input as height information, and 1.5m, which is the maximum field of view that the camera can recognize forward, can be input as distance information. Through this, the robot (300) equipped with the passage recognition device (100) can stably recognize the passage that can be moved in the vineyard. In a chili pepper field environment, the passage recognition device (100) performs preliminary work to recognize a narrow passage between the ridges, in which case 0.15m from the ground to the height of the chili pepper is input as height information, and 1.5m, which is the maximum field of view distance at which the camera can recognize the front, can be input as distance information.

[0075] Meanwhile, height information corresponding to the height of the tree and the height of the ridge from the ground may not be constant, but accuracy can be maintained because a single deep learning-based model (53) can adapt to the changing environment. For example, the deep learning model can learn and apply various height changes in different environments such as orchards, vineyards, and chili pepper fields. This can help the robot (300) to explore a movable path and determine the direction of movement for the movable path.

[0076] Here, when setting the 'U'-shaped passageway characteristics (51), the height information is set first because the element most affected by the environment is the height information. This is not limited to height information, and in some cases, information about the width may be entered as prior information when setting the 'U'-shaped passageway characteristics (51).

[0078] FIG. 3 is a drawing for explaining a passage recognition device according to an embodiment of the present invention. FIG. 4 is a drawing for explaining a control unit of a passage recognition device according to an embodiment of the present invention.

[0079] Here, the drawings illustrated in FIGS. 3 and 4 are merely exemplary, and the concept of the present invention is not to be interpreted as being limited to that illustrated in FIGS. 3 and 4. Two or more components described below may be combined into a single component, or a single component may be divided into two or more components according to more detailed functions. This will be described in more detail later with reference to FIGS. 3 and 4.

[0080] Referring to FIG. 3, a passage recognition device (100) according to an embodiment of the present invention includes a communication unit (110), an input unit (130), a control unit (150), and a storage unit (190).

[0081] The communication unit (110) communicates with the robot (300) and the image capturing device (200). The communication unit (110) receives image information including a movable passage from the image capturing device (200). The communication unit (110) receives image information of the environment to be applied from the image capturing device (200) and transmits information related to a 'U'-shaped moving passage, which includes 'U'-shaped passage characteristics (51) in the received image data, to the robot (300).

[0082] The storage unit (170) stores a program or algorithm for driving the passage recognition device (100). In detail, the storage unit (190) stores acquired image information and stores information that is divided, searched, and generated during the process of setting a 'U'-shaped movement passage and determining the direction of movement. The storage unit (190) may include at least one storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and an optical disk.

[0083] As described above, the passage recognition device (100) according to the embodiment of the present invention can store a plurality of data through the storage unit (170) via several steps from the input unit (130).

[0084] The control unit (150) performs overall control of the passage recognition device (100). As shown in FIG. 4, the control unit (150) includes a passage search unit (153) including a single model (53), a direction determination unit (155), and may further include a passage characteristic setting unit (151).

[0085] When image information of an environment to be applied is input to the control unit (150) of the passage recognition device (100) according to an embodiment of the present invention, the control unit (150) sets a ‘U’-shaped passage characteristic (51) composed of a predefined floor surface (515) and a wall surface (511) using prior information that includes at least one of the maximum field of view distance information of a camera, height information from the floor, and depth information of the image information. More specifically, the passage characteristic setting unit (151) of the control unit (150) sets a ‘U’-shaped passage characteristic (51) for the input image information in order to apply a single model (53) that searches for the shape of the passage characteristic (51) of a movable space to the image information. Alternatively, the passage characteristic setting unit (151) sets a ‘U’-shaped passage characteristic (51) composed of a floor surface (515) and a wall surface (511) using the input prior information when at least one of the height information from the floor, the field of view information of the camera mounted on the robot (300), and the depth information of the image information is input as prior information to search for a movable passage included in the image information. As described above, the passage characteristic (51) may be set by the user terminal (500) before the image information of the environment to be applied is input, but is not limited thereto, and the passage characteristic (51) having a ‘U’-shaped form may also be set by the passage characteristic setting unit (151) of the control unit (150). When a passage characteristic (51) having a 'U' shape is set by the characteristic setting unit (151), the passage search unit (153) of the control unit (150) can help to perform a search for a movement passage according to a change in the shape of the movement passage by implementing a single model (53) independent of the operating environment based on the passage characteristic (51). The passage search unit (153) may further include a single model (53).

[0086] In this way, when video information to which a moving body is to be applied is input to the passage search unit (153), the input video information is divided into multiple areas at predetermined distances. Subsequently, the path that the robot (300) can move along is analyzed to help accurately recognize passages that can be moved at multiple distances. To this end, the passage search unit (153) implements a single model (53) independent of the operating environment based on the 'U'-shaped passage characteristic (51) and applies it to the video information divided by each distance. In this way, the passage search unit (153) applies the single model (53) that searches for a specific shape of the movable space to the video information to search for a 'U'-shaped movement passage that has the shape of a 'U'-shaped passage. Here, each distance can be set based on the movement speed of the robot (300) and the 'U'-shaped passage characteristic (51).

[0087] In detail, the control unit (150) may further include a preprocessing unit (35). The preprocessing unit (35) may transform each of the divided multiple regions through at least one of blur, expand, and erode. The preprocessing unit (35) applies various image processing techniques to each of the divided multiple regions. The transformation by the preprocessing unit (35) may include at least one of blur, expand, and erode. For example, when performing blur processing on each of the divided multiple regions, noise in the image can be removed to clarify the boundaries of the 'U'-shaped path characteristics (51). For example, the expand processing can expand the size of an object to emphasize small objects or detailed structures and ensure the continuity of the 'U'-shaped path characteristics (51). The transformation by erode processing can reduce the size of an object to remove noise and make the boundaries of the object even clearer. This load processing increases the reliability of the 'U'-shaped passage characteristic (51) by removing unnecessary details. Subsequently, the passage search unit (153) can search for a movable passage by applying a single model (53) that searches for the shape of the 'U'-shaped passage characteristic (51) of the movable space to multiple areas divided by distance.

[0089] FIG. 5 is a diagram showing the characteristics of a ‘U’-shaped passage applied by distance and a method for determining the direction of curvature thereto according to an embodiment of the present invention, and specifically, a diagram for explaining the direction of curvature for a straight passage. FIG. 6 is a diagram showing the characteristics of a ‘U’-shaped passage applied by distance and a method for determining the direction of curvature thereto according to an embodiment of the present invention, and specifically, a diagram for explaining the direction of curvature for a curved passage.

[0090] When a 'U'-shaped movement path is established by connecting neighboring 'U'-shaped path characteristics (51) among a plurality of 'U'-shaped path characteristics (51) applied by distance by the path search unit (153), the direction determination unit (155) of the control unit (150) extracts the center point of the floor surface (515) existing within the plurality of 'U'-shaped path characteristics (51) applied by distance to a plurality of divided areas. Subsequently, the direction determination unit (155) sets the curvature direction for the 'U'-shaped movement path by connecting neighboring center points among the extracted plurality of center points, and can determine the curvature direction for the path that can be moved from the current position of the robot (300) based on the set curvature direction.

[0091] Referring to FIGS. 5 and 6, the direction determining unit (155) can recognize the direction of curvature through the change in the shape of the ‘U’-shaped passage characteristic (51) applied according to distance in straight passages and curved passages. For example, based on the change in the shape of the ‘U’-shaped passage characteristic (51) applied according to distance by the passage search unit (153), if the shape of the passage characteristic (51) is maintained constant, specifically if the height of both walls (511) is maintained constant, the direction determining unit (155) determines the corresponding movement passage as a straight path. Subsequently, the direction determining unit (155) extracts the center point of the floor surface (515) of the ‘U’-shaped movement passage according to distance and connects neighboring center points to each other so that the movement target point of the robot (300) is subsequently controlled based on the center point of the floor surface (515).

[0092] That is, the robot (300) can drive along a straight path when the height of both walls (511) is maintained constant by continuously analyzing the depth image and updating the 'U'-shaped passage characteristic (51). Alternatively, the direction determining unit (155) can determine that there is a curvature to the left when only the height of the left wall at the very front is recognized based on the shape change of the 'U'-shaped passage characteristic (51) recognized as the distance increases, or when the height of the left wall appears more clearly than the height of the right wall.

[0093] That is, the direction determining unit (155) can determine information about the curvature of the movement path by comparing the shape of the ‘U’-shaped passage characteristic (51) recognized at the minimum distance with the change in the ‘U’-shaped passage characteristic (51) recognized as the distance increases.

[0095] FIG. 7 is a flowchart for explaining a passage recognition method performed by a passage recognition device according to an embodiment of the present invention.

[0096] In step S110 of the passage recognition method performed by the passage recognition device (100), when image information of the environment to which a moving object is applied is input using an RGB-D camera, the control unit (150) of the passage recognition device (100) sets the passage characteristics (51) to apply a single model (53) that searches for the shape of the passage characteristics (51) of the movable space to the image information. Specifically, the control unit (150) analyzes the image information input through the RGB-D camera to identify the passage characteristics (51) of the movable passage in the input image information (S110). In addition, the control unit (150) can identify the movable path and obstacles (S110). The image information passes through a preprocessing unit (35) to remove noise and extract only the necessary parts for analysis (S110). Here, the passageway characteristic (51) can be defined as a ‘U’-shaped passageway characteristic (51) composed of a floor surface (515) and a wall surface (511) so that it can be applied independently to the operating environment (S110). That is, the passageway characteristic (51) can be implemented in a ‘U’ shape including the floor surface (515) and the wall surface (511), and the ‘U’-shaped passageway characteristic (51) can be set by using at least one of the information among height information from the floor in the application environment and the maximum field of view distance information of the camera mounted on the passageway recognition device (100) based on the movement speed as prior information (S110).

[0097] In step S120, the control unit (150) of the passage recognition device (100) divides the image information containing a movable passage into multiple regions at set distances. Considering the maximum field of view distance that the camera can see and the movement speed, the control unit (150) of the passage recognition device (100) applies a predefined single model (53) to the image information to divide the image by distance so that the image is divided at appropriate distance intervals (S120). For example, if the maximum field of view distance of the camera based on the movement speed is 1.5m, the control unit (150) divides the image by distance at intervals of 0.5m (S120).

[0098] In step S130, the control unit (150) of the passage recognition device (100) searches for a 'U'-shaped movement passage having the shape of a 'U'-shaped passage by applying a single model (53) that searches for a specific shape of a movable space to multiple image information divided by distance (S130). Specifically, the control unit (150) applies a single model (53) configured to search for the shape of a 'U'-shaped passage characteristic (51) of a movable space to multiple areas divided by distance (S130). The control unit (150) extracts the 'U'-shaped passage characteristic (51) applied to the multiple divided image information, connects neighboring 'U'-shaped passage characteristics (51) among the extracted multiple 'U'-shaped passage characteristics (51), and sets the connected 'U'-shaped passage characteristic (51) as a 'U'-shaped movement passage (S130).

[0099] Subsequently, in step S140, when a 'U'-shaped movement path is established by connecting neighboring 'U'-shaped path characteristics (51) by the control unit (150), the control unit (150) of the path recognition device (100) extracts the center point of the floor surface (515) of the 'U'-shaped path characteristics (51) for each distance existing within the 'U'-shaped movement path. The control unit (150) sets the curvature direction for the 'U'-shaped movement path by connecting neighboring center points among the extracted multiple center points of the floor surface (515), and determines the curvature direction for the path that can be moved from the current position of the robot (300) based on the set curvature direction (S140). Here, the control unit (150) can determine information regarding the curvature of the movement path by comparing the shape of the 'U'-shaped path characteristic (51) recognized at the minimum distance with the change in the 'U'-shaped path characteristic (51) recognized as the distance increases (S140).

[0100] The control unit (150) of the passage recognition device (100) recognizes the direction of curvature of the movement path by connecting the center points of the floor surface (515) of the recognized 'U'-shaped movement path for each distance (S140). For example, if the center points are in a straight line, the control unit (150) determines that the 'U'-shaped movement path is a straight path and sets the robot (300) to move along the straight path based on the center points (S140). Alternatively, if the center points are not in a straight line and form a curved shape, the control unit (150) of the passage recognition device (100) determines that the 'U'-shaped movement path is a curved path and sets the robot (300) to move along the curved path based on the center points (S140).

[0102] FIG. 8 is a diagram illustrating a method for determining a direction of movement based on a current position by a robot according to an embodiment of the present invention. Referring to FIG. 8, the process of determining the direction of movement performed by a robot (300) according to an embodiment of the present invention involves setting a ‘U’-shaped passageway characteristic (51) including a floor surface (515) and a wall surface (511) using at least one of information including maximum field of view distance information considering the movement speed of the robot (300), height information from the floor, and depth information of image information, and applying a single model (53) independent of the operating environment to each area divided by distance based on the passageway characteristic (51), and determining the direction of movement by connecting the center points of the floor surface (515) of the ‘U’-shaped passageway characteristic (51) adjacent by distance.

[0103] In detail, in step S210 of the method for determining the direction of movement performed by a robot (300) according to an embodiment of the present invention, when image information of the environment to be applied is input using an RGB-D camera, the robot (300) divides the image into appropriate distance intervals considering the maximum field of view distance that the camera can see and the movement speed (S210). For real-time image information including a movable passage, the image is divided into multiple regions by dividing it at predetermined distances (S210).

[0104] In step S220, the robot (300) applies a single model (53) to multiple divided areas to extract the center point of the floor surface (515) included in each of the neighboring 'U'-shaped passageway characteristics (51). Specifically, the robot (300) applies a single model (53) that explores the shape of the passageway characteristics (51) of the movable space to each of the multiple image information divided by distance to extract the center point of the floor surface (515) in each area (S220). After extracting the center point of the floor surface (515) of the 'U'-shaped passageway characteristics (51) applied by distance, the robot (300) recognizes the curvature direction of the movement path based on the extracted center point of the floor surface (515) (S220). The robot (300) may not implement some of the shapes of the 'U'-shaped passageway characteristics (51) recognized at the minimum distance based on the movement speed (S220). For example, in the case of a sharp turn to the right or left, the robot (300) divides the area into multiple regions by distance, taking into account the movement speed as the distance increases, and even if the divided multiple regions are applied to a single model (53), there may be areas where the right wall (511) or left wall (511) of the 'U'-shaped passage characteristic (51) is not recognized (S220). In this situation, the robot (300) divides the area into multiple regions by distance based on the movement speed and cannot extract the center point of the floor surface (515) (S220). In this situation, the robot (300) divides the area into multiple regions by distance based on the movement speed and extracts the center point of the floor surface (515) from each region. If the number of center points of the floor surface (515) extracted from the multiple regions divided by distance is less than or equal to a preset number, the robot (300) determines that it is a sharp turn to the right or left, and the robot (300) reduces the speed in the corresponding region to below a threshold speed.Alternatively, when the robot (300) divides the space into multiple areas based on distance according to the movement speed and applies a single model (53) that searches for the shape of the passage characteristic (51) of the movable space to the divided multiple areas, if the speed at which the left wall (511) of the 'U'-shaped passage characteristic (51) disappears is greater than or equal to a preset speed, the robot (300) can reduce the driving speed to below a threshold speed (S220).

[0105] Subsequently, in step S230, the direction of movement is determined by connecting the center points of the bottom surface (515) of the 'U'-shaped passageway characteristics (51) extracted from each of the multiple divided regions applied according to distance. The robot (300) sets the curvature direction of the movement path by overlapping the center points of the bottom surface (515) of the 'U'-shaped passageway characteristics (51) extracted from each of the multiple divided regions applied according to distance and connecting adjacent center points (S230). Based on the set curvature direction, the robot (300) determines the curvature direction of the passageway that can be moved from the current position (S230). The robot (300) determines the curvature information of the movement path by comparing the shape of the 'U'-shaped passageway characteristics (51) recognized at the minimum distance with the change in the shape of the 'U'-shaped passageway characteristics (51) recognized as the distance increases (S230). For example, if the center points are on a straight line, the robot (300) determines that the 'U'-shaped movement path is a straight path and sets it to move along the straight path based on the center point. Meanwhile, in cases where the center points do not exist on a straight line but form a curved shape, the robot (300) determines the ‘U’-shaped passageway as a curved path and sets it to move along a curved path based on the center point. That is, the robot (300) can determine information about the curvature of the passageway by comparing the shape of the ‘U’-shaped passageway characteristic (51) recognized at the minimum distance with the change in the shape of the ‘U’-shaped passageway characteristic (51) recognized as the distance increases (S230).

[0107] According to an embodiment of the present invention, by setting a 'U'-shaped passage characteristic that can be utilized in various operating environments of a robot and searching for a movable passage using the 'U'-shaped passage characteristic, the effect of determining a movement passage without resetting the algorithm or performing additional work according to the environment is achieved.

[0108] According to an embodiment of the present invention, by utilizing the characteristics of a 'U'-shaped passage to quickly and accurately recognize a movable passage, it has the effect of facilitating the movement of a robot in environments with complex structures, such as orchards or farmlands.

[0109] According to an embodiment of the present invention, by utilizing the characteristics of a 'U'-shaped passage, the curvature direction of the movement passage is recognized, and the movement path of the robot is controlled in real time.

[0110] According to an embodiment of the present invention, by recognizing the direction of curvature using the characteristics of a 'U'-shaped passage, the driving stability of the robot is enhanced and the ability to avoid obstacles is improved.

[0112] FIG. 9 is a drawing showing a computing device according to an embodiment of the present invention. The computing device (TN100) of FIG. 9 may be a device described in the present specification (e.g., a passage recognition device (100), an image capturing device (200), a robot (300), a user terminal (500), a passage recognition system (10), etc.).

[0113] In the embodiment of FIG. 9, the computing device (TN100) may include at least one processor (TN110), a transceiver (TN120), and a memory (TN130). Additionally, the computing device (TN100) may further include a storage device (TN140), an input interface device (TN150), an output interface device (TN160), etc. The components included in the computing device (TN100) may be connected by a bus (TN170) to communicate with each other.

[0114] The processor (TN110) can execute a program command stored in at least one of the memory (TN130) and the storage device (TN140). The processor (TN110) may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. The processor (TN110) may be configured to implement the procedures, functions, and methods described in relation to embodiments of the present invention. The processor (TN110) can control each component of the computing device (TN100).

[0115] Each of the memory (TN130) and the storage device (TN140) can store various information related to the operation of the processor (TN110). Each of the memory (TN130) and the storage device (TN140) may be composed of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (TN130) may be composed of at least one of read-only memory (ROM) and random access memory (RAM).

[0116] The transmitting and receiving device (TN120) can transmit or receive wired or wireless signals. The transmitting and receiving device (TN120) can be connected to a network to perform communication. And the computing device (TN100) can have a single antenna or multiple antennas.

[0118] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by a person skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention. Explanation of the symbols

[0120] 10: Pathway Recognition System 100: Pathway recognition device 200: Video recording device 400 : Network 300 : Robot 130 : Input section 110 : Communication section 150 : Control unit 170 : Storage unit 151 : Passage Characteristics Setting Section 153: Pathway Search Unit 155: Direction Determination Unit 500 : User terminal

Claims

Claim 1 A passage recognition device comprising: a passage search unit that, when image information of an environment to which a moving body is to be applied is input, sets passage characteristics for said image information and searches for a movable passage by applying a single model that searches for a shape corresponding to said passage characteristics to said image information; and a direction determination unit that determines the direction of movement of said moving body for said movable passage; wherein the passage characteristics are 'U'-shaped passage characteristics composed of a floor surface and a wall surface; the passage search unit divides said image information into multiple regions at predetermined distances and searches for a movable passage by applying said 'U'-shaped passage characteristics to each of said multiple regions; and when a movable passage is found by applying said 'U'-shaped passage characteristics to each of said multiple regions, the direction determination unit extracts center points of floor surfaces existing within a plurality of 'U'-shaped passage characteristics at each distance applied to said multiple regions, and determines the curvature direction for said movable passage by connecting adjacent center points among said extracted center points. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 A passage recognition device according to claim 1, wherein a passage characteristic setting unit for setting the passage characteristic is additionally provided, and wherein the passage characteristic setting unit sets a 'U'-shaped passage characteristic composed of a floor surface and a wall surface for the single model using the input prior information when at least one of height information from the floor, field of view information of a camera mounted on the moving body, and depth information of the image information is input as prior information to search for the movable passage for the image information. Claim 6 A passage recognition device comprising: a passage search unit that, when image information of an environment to which a moving body is to be applied is input, sets passage characteristics for said image information and searches for a movable passage by applying a single model that searches for a shape corresponding to said passage characteristics to said image information; and a direction determination unit that determines the direction of movement of said moving body for said movable passage; wherein, when a 'U'-shaped movement passage is set by connecting adjacent 'U'-shaped passage characteristics among a plurality of 'U'-shaped passage characteristics applied according to distance, said direction determination unit extracts the center point of the floor surface of said 'U'-shaped passage characteristics included in said 'U'-shaped movement passage, sets the curvature direction for said 'U'-shaped movement passage by connecting adjacent floor surface center points among said extracted floor surface center points, and determines the direction of movement based on the current position of said moving body based on said set curvature direction. Claim 7 A passage recognition device according to claim 1, wherein the passage search unit further comprises a preprocessing unit that modifies each of the divided plurality of regions through at least one of blur, expand, and erode. Claim 8 A passage recognition method performed by a passage recognition device, comprising: a step of setting passage characteristics to apply a single model that searches for a shape corresponding to the passage characteristics of a movable space to the image information when image information of an environment to which a moving body is applied is input; a step of dividing the image information into multiple regions at predetermined distances; a step of searching for a 'U'-shaped movement passage by applying the single model that searches for the shape of a 'U'-shaped passage characteristic to the multiple divided regions; and a step of determining a direction of movement based on the shape change of the 'U'-shaped movement passage recognized by distance for the 'U'-shaped movement passage; wherein the step of determining the direction of movement comprises extracting center points of floor surfaces existing within the multiple 'U'-shaped passage characteristics applied by distance to the multiple divided regions, and determining the curvature direction for the movable passage by connecting adjacent center points among the extracted multiple center points. Claim 9 delete Claim 10 A robot that communicates with a passage recognition device according to claim 1, and determines the direction of movement for the movable passage based on the current position, based on the shape change of the movable passage recognized by distance.

Images