Robot using moving walkway and method of controlling same

Abstract

There is provided a robot including: a camera; a driving part including a plurality of wheels and a suspension for connecting each of the plurality of wheels to a main body of the robot; and a processor. A processor that identifies a moving direction of a moving channel and a moving speed of the moving channel based on a plurality of images for the moving channel received from a camera after identifying that the robot is located in an entrance area of the moving channel; a control unit that controls the driving unit so that the robot points in an entry direction corresponding to the recognized movement direction; controlling a suspension connected to at least one of the plurality of wheels such that the robot tilts at an angle corresponding to the identified movement speed; and controlling the driver so that the inclined robot moves according to the entering direction to enter the moving channel.

Description

Technical Field

[0001] Apparatus and methods consistent with the present disclosure relate to a robot and a method of controlling the same. More specifically, the present disclosure relates to a robot using a moving channel and a method of controlling the same. Background Art

[0002] Recently, with the development of electronic technology, robots are being applied to various industrial fields. With the development of object recognition technology, robots can accurately distinguish various objects, and with the development of autonomous driving technology, robots can drive safely without interfering with human traffic in the driving space. It can be seen that robots such as service robots that provide food ordered by users in restaurants and guide robots that guide users on the way at airports, large markets or other places are used in various fields and in various ways.

[0003] As robots are used in various fields, the types and sizes of spaces in which robots travel have also diversified. In the past, robots could only travel in narrow and confined spaces, but now they can travel in wider and more open spaces. In the case of large buildings including multiple floors (such as large markets and airports), robots can move between floors in large buildings by using moving passages (e.g., inclined moving passages). However, when the robot uses the moving passage, the robot needs to take a suitable posture in consideration of the moving speed or tilt angle of the moving passage.

[0004] Specifically, unlike stairs fixed to the ground, since the moving passage moves, the robot that gets on the moving passage needs to take a stable posture in consideration of the moving speed of the moving passage. In addition, unlike escalators in which stairs are formed by a plate parallel to the ground and move, the moving passage that connects the travel spaces of different floors has an inclined angle, so the robot that gets on the moving passage needs to take a stable posture suitable for the inclined angle of the moving passage.

[0005] However, after the conventional robot gets on the moving channel, it can take a stable posture by detecting the moving speed and tilt angle of the moving channel. However, this method has the following problems: the robot may not be able to predict the moving speed and tilt angle of the moving channel and take a stable posture in advance. Summary of the invention

[0006] The disclosed embodiments relate to a robot using a moving walkway and a method of controlling the same.

[0007] A robot may include: a camera; a driver including a plurality of wheels and a suspension for connecting each of the plurality of wheels to a body of the robot; and a processor. The processor may be configured to: after the processor recognizes that the robot is located in an entrance area of ​​a moving passage, recognize a moving direction of the moving passage and a moving speed of the moving passage based on a plurality of images of the moving passage received from the camera; control the driver so that the robot faces an entry direction corresponding to the recognized moving direction; control a suspension connected to at least one of the plurality of wheels so that the robot tilts at an angle corresponding to the recognized moving speed; and control the driver so that the tilted robot moves in the entry direction and enters the moving passage.

[0008] The processor may also be configured to: when the robot is located in the entrance area, identify boundary lines between multiple plates of the moving channel based on the multiple images received from the camera; identify a moving direction of the moving channel based on a normal vector of the identified boundary line; and identify a moving speed of the moving channel based on a moving speed of the identified boundary line.

[0009] The processor may be configured to: after the processor identifies that the robot has entered the moving channel, attempt to identify multiple plates of the moving channel based on the multiple images of the moving channel received from the camera; identify an inclination angle of the moving channel based on a normal vector of at least one of the identified multiple plates; and control the suspension connected to at least one of the multiple wheels based on the identified inclination angle so that the robot stands upright.

[0010] The processor may be configured to identify the at least one of the plurality of panels based on a moving speed of the moving channel and a response speed of the suspension.

[0011] The robot may further include a sensor. The processor may be further configured to: based on the processor not recognizing the plurality of boards based on the plurality of images received from the camera when the robot enters the moving channel, control the suspension connected to the at least one wheel among the plurality of wheels based on a sensing value received from the sensor, so that the robot stands upright.

[0012] After the processor identifies that the robot is within a predetermined distance from an exit area of ​​the moving channel, the processor may be configured to re-identify the moving direction of the moving channel based on the multiple images acquired, control the driver so that the robot faces the exit direction corresponding to the re-identified moving direction, and control the driver so that the robot moves in the exit direction and leaves the moving channel.

[0013] After the processor recognizes that there is an object in the exit area, the processor may control the driver to move the robot in a direction opposite to the exit direction.

[0014] When the identified tilt angle is within a predetermined angle range, the processor may be configured to identify that the robot is within the predetermined distance from the exit area of ​​the moving passage.

[0015] The processor may also be configured to: when the robot is located in the entrance area, identify the width of the moving channel based on the multiple images received from the camera, determine the entrance position of the moving channel of the robot based on the identified width of the moving channel, and control the drive so that the robot faces the entry direction corresponding to the identified moving direction at the determined entrance position.

[0016] A method for controlling a robot includes: after recognizing that the robot is located in an entrance area of ​​a moving passage, recognizing a moving direction of the moving passage and a moving speed of the moving passage based on a plurality of images of the moving passage received from a camera; controlling a driver of the robot so that the robot faces an entry direction corresponding to the recognized moving direction; controlling a suspension of the driver so that the robot tilts at an angle corresponding to the recognized moving speed, wherein the suspension is connected to at least one wheel of a plurality of wheels included in the driver; and controlling the driver so that the tilted robot moves in the entry direction and enters the moving passage. Each of the plurality of wheels is connected to a body of the robot through the suspension.

[0017] Identifying the moving direction of the moving channel and the moving speed of the moving channel may include: when the robot is located in the entrance area, identifying a boundary line between multiple plates of the moving channel based on the multiple images received from the camera; identifying the moving direction of the moving channel based on a normal vector of the identified boundary line; and identifying the moving speed of the moving channel based on the moving speed of the identified boundary line.

[0018] The method may further include: after identifying that the robot enters the moving channel, attempting to identify multiple plates of the moving channel based on the multiple images of the moving channel received from the camera; identifying an inclination angle of the moving channel based on a normal vector of at least one plate among the identified multiple plates; and controlling the suspension connected to the at least one wheel among the multiple wheels based on the identified inclination angle so that the robot stands upright.

[0019] The method may further include identifying the at least one panel among the plurality of panels based on a moving speed of the moving channel and a response speed of the suspension.

[0020] The method may further include: after the robot cannot identify the multiple boards based on the multiple images received from the camera when entering the moving channel, controlling the suspension connected to at least one of the multiple wheels based on a sensing value obtained by a sensor of the robot so that the robot stands upright.

[0021] The method may further include: after identifying that the robot is located within a predetermined distance from an exit area of ​​the moving channel, re-identifying the moving direction of the moving channel based on the multiple images acquired; controlling the driver so that the robot faces the exit direction corresponding to the re-identified moving direction; and controlling the driver so that the robot moves in the exit direction and leaves the moving channel.

[0022] The method may further include, after recognizing that an object exists in the exit area, controlling the driver to move the robot in a direction opposite to the exit direction.

[0023] The method may further include recognizing that the robot is within the predetermined distance from the exit area of ​​the moving passage when the recognized tilt angle is within a predetermined angle range.

[0024] The method may further include: when the robot is located in the entrance area, identifying the width of the moving channel based on the multiple images received from the camera; determining the entrance position of the moving channel of the robot based on the identified width of the moving channel; and controlling the driver so that the robot faces the entry direction corresponding to the identified moving direction at the determined entrance position.

[0025] A non-transitory computer-readable recording medium may include a program for executing a method of controlling a robot. The method may include: after recognizing that the robot is located in an entrance area of ​​a moving channel, identifying a moving direction of the moving channel and a moving speed of the moving channel based on a plurality of images of the moving channel received from a camera; controlling a driver of the robot so that the robot faces an entry direction corresponding to the recognized moving direction; controlling a suspension of the driver so that the robot tilts at an angle corresponding to the recognized moving speed, wherein the suspension is connected to at least one of a plurality of wheels included in the driver; and controlling the driver so that the tilted robot moves in the entry direction and enters the moving channel. Each of the plurality of wheels may be connected to a body of the robot through the suspension.

[0026] Identifying the moving direction of the moving channel and the moving speed of the moving channel may include: when the robot is located in the entrance area, identifying a boundary line between multiple plates of the moving channel based on the multiple images received from the camera; identifying the moving direction of the moving channel based on a normal vector of the identified boundary line; and identifying the moving speed of the moving channel based on the moving speed of the identified boundary line. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic illustration of a robot using a moving walkway according to one or more embodiments of the present disclosure.

[0028] Figure 2 is a schematic configuration diagram of a robot according to one or more embodiments of the present disclosure.

[0029] Figure 3 is an exemplary diagram illustrating a method of identifying a moving direction and a moving speed of a moving walkway according to one or more embodiments of the present disclosure.

[0030] Figure 4 is an example diagram of a method of identifying a moving direction and a moving speed of a moving walkway based on a boundary line between boards according to one or more embodiments of the present disclosure.

[0031] Figure 5 is an example diagram showing how to determine a boarding direction of a robot based on a moving direction of a moving walkway according to one or more embodiments of the present disclosure.

[0032] Figure 6 is an example diagram illustrating determining a tilt angle of a robot based on a moving speed of a moving walkway according to one or more embodiments of the present disclosure.

[0033] Figure 7is an example diagram showing a robot boarding a moving walkway recognizing a plurality of boards according to one or more embodiments of the present disclosure.

[0034] Figure 8 is an exemplary diagram of a method for identifying an inclination angle of a moving walkway based on a normal vector of a board according to one or more embodiments of the present disclosure.

[0035] Fig. 9 is an example diagram of a method of controlling a suspension of at least one wheel of a robot based on an identified tilt angle according to one or more embodiments of the present disclosure.

[0036] Fig.10 is a diagram for describing recognition of a departure area and a distance between the departure area and a robot based on an image acquired by a camera according to one or more embodiments of the present disclosure.

[0037] Fig.11 is a diagram for explaining that, when an object exists in a departure area, a driver is controlled so that the robot moves in a direction opposite to a departure direction according to one or more embodiments of the present disclosure.

[0038] Fig.12 is a detailed configuration diagram of a robot according to one or more embodiments of the present disclosure.

[0039] Fig.13 is a flowchart schematically illustrating a method of controlling a robot to board a moving walkway according to one or more embodiments of the present disclosure.

[0040] Fig.14 is a flowchart schematically illustrating a method of controlling a robot when the robot boards a moving walkway according to one or more embodiments of the present disclosure.

[0041] Fig.15 is a flowchart schematically illustrating a method of controlling a robot to leave a moving walkway according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] Since the present disclosure may be variously modified and has several exemplary embodiments, specific exemplary embodiments will be shown in the drawings and described in detail in the detailed description. However, it should be understood that the present disclosure is not limited to specific exemplary embodiments, but includes all modifications, equivalents and replacements according to the exemplary embodiments. Throughout the drawings, similar components will be represented by similar reference numerals.

[0043] In describing the present disclosure, when it is decided that a detailed description of a known function or configuration related to the present disclosure may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted.

[0044] In addition, the following exemplary embodiments may be modified in several different forms, and the scope and spirit of the present disclosure are not limited to the following exemplary embodiments. On the contrary, these exemplary embodiments make the present disclosure thorough and complete, and are provided to fully convey the spirit of the present disclosure to those skilled in the art.

[0045] The terms used in the present disclosure are only used to describe specific exemplary embodiments rather than to limit the scope of the present disclosure. Unless the context clearly indicates otherwise, a singular form is intended to include a plural form.

[0046] In the present disclosure, the expressions “having”, “may have”, “including”, “may include”, etc. indicate the existence of corresponding features (e.g., values, functions, operations, components such as parts, etc.), and do not exclude the existence of additional features.

[0047] In the present disclosure, the expressions "A or B", "at least one of A or / and B", "one or more of A or / B", etc. may include all possible combinations of items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may indicate: 1) a case where at least one A is included; 2) a case where at least one B is included; 3) a case where both at least one A and at least one B are included.

[0048] The expressions "first", "second", "1st", "2nd", etc. used in the present disclosure may indicate various components, regardless of the order and / or importance of the components, and will only be used to distinguish one component from other components and will not limit the corresponding components.

[0049] When referring to any component (e.g., a first component) being (operably or communicatively) coupled to / coupled to another component (e.g., a second component) or connected to another component (e.g., a second component), it should be understood that any component is directly coupled to another component or may be coupled to another component through other components (e.g., a third component).

[0050] On the other hand, when it is mentioned that any component (e.g., a first component) is "directly coupled" or "directly connected" to another component (e.g., a second component), it should be understood that other components (e.g., a third component) do not exist between any component and the other component.

[0051] The expression “~ is configured (or set) to” used in the present disclosure may be replaced by the expression “suitable for”, “having the ability to…”, “~ is designed to”, “~ is adapted to”, “~ is made to”, or “~ is capable of” according to circumstances. The term “~ is configured (or set) to” may not necessarily mean “specially designed to” in hardware.

[0052] In contrast, the expression “~the device is configured to” may mean that the device, together with other devices or components, is “capable of…” For example, “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for performing the corresponding operations or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can perform the corresponding operations by executing one or more software programs stored in a storage device.

[0053] In an exemplary embodiment, a "module" or "unit" may perform at least one function or operation and may be implemented by hardware or software, or by a combination of hardware and software. In addition, multiple "modules" or multiple "devices" may be integrated into at least one module and may be implemented by at least one processor, except for "modules" or "devices" that need to be implemented by specific hardware.

[0054] Various elements and regions in the drawings are schematically shown. Therefore, the spirit of the present disclosure is not limited by the relative sizes or intervals shown in the drawings.

[0055] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure pertains can be easily practiced.

[0056] Figure 1 is a schematic illustration of a robot using a mobile channel according to one or more embodiments of the present disclosure.

[0057] A moving channel refers to a structure or mechanical device designed to be installed as a conveyor belt in a travel space so that, for example, pedestrians, goods, and / or robots can move automatically. A moving channel may be referred to as a moving walkway, a moving sidewalk, a moving pedestrian walkway, a moving walkway, and / or an automatic conveyor belt. In this specification, for the convenience of description, it is referred to as a moving channel.

[0058] The moving passage 200 used by the robot 100 may include a horizontal moving passage that may be flush with a floor surface to move within the same floor, and an inclined moving passage that may be formed to have a certain slope to move between floors.

[0059] The robot 100 can be implemented as various electronic devices, such as a robot capable of performing air purification operations while traveling in a building space, a housekeeping support robot capable of performing operations such as arranging clothes and washing dishes while traveling in a home space, or a security robot capable of performing security while traveling in a building space.

[0060] However, the robot 100 is not limited thereto and may be implemented as various movable robots. For example, the robot 100 may be implemented as a robot vacuum cleaner capable of performing cleaning operations while traveling in a home space, an autonomous driving vehicle capable of performing operations instead of humans, or an automatic guided vehicle capable of moving items to a destination.

[0061] Reference Figure 1 , the robot 100 uses the inclined moving passage 200. The robot 100 can move between the travel spaces 1 and 2 on different floors through the inclined moving passage 200.

[0062] In this case, the robot 100 may recognize the moving speed and moving direction of the moving path 200 before boarding the moving path 200 (①), and recognize the posture of the robot based on the moving speed and moving direction. The recognized posture of the robot may be a posture recognized based on the moving speed and moving direction of the moving path 200 so that the robot 100 can stably board the moving path 200. Therefore, the robot 100 may not fall when boarding the moving path 200, or reduce the impact applied to the object or food mounted on the robot 100.

[0063] In addition, the robot 100 may recognize the inclination angle of the moving passage 200, and recognize the posture of the robot based on the inclination angle. The recognized posture of the robot may be a posture recognized based on the inclination angle of the moving passage 200 so that the robot 100 can stably board the moving passage 200. More specifically, the posture recognized based on the inclination angle of the moving passage 200 may be a posture in which the robot 100 keeps standing so that the robot 100 stands upright with a workpiece mounted thereon. Therefore, the robot 100 may more safely transfer an object or food mounted on the robot 100 when boarding the moving passage 200 moving at an inclination angle.

[0064] In addition, the robot 100 may identify the moving direction of the moving passage 200 before leaving the moving passage 200 (③), and identify the exit direction (e.g., exit direction) of the robot 100 based on the identified moving direction. The identified exit direction may be a direction that matches the moving direction of the moving passage 200 so that the robot 100 can stably leave the moving passage 200. Therefore, when the robot 100 leaves the moving passage 200, it can leave in the correct direction.

[0065] The state (①) before the robot 100 boards the moving channel 200 is set as the boarding preparation mode, the state of the robot 100 in the state of having boarded the moving channel 200 is set as the boarding mode, and the state (③) before the robot leaves the moving channel 200 is set as the leaving mode, and the operation method of the robot 100 in each mode can be distinguished.

[0066] Hereinafter, the robot 100 using the moving channel 200 will be described in detail.

[0067] Figure 2 is a schematic configuration diagram of a robot 100 according to one or more embodiments of the present disclosure. Figure 2 , the robot 100 includes a camera 110 , a driver 120 , and a processor 130 .

[0068] The robot 100 may be divided into a head and a main body. For example, the head of the robot 100 may include a camera 110 or a sensor to perform a function of acquiring an image of an object around the robot 100 or detecting the object. The main body of the robot 100 may be connected to a driver 120 to move the robot 100 or perform a function of supporting a frame of the robot 100. In this case, the head may be connected to a rotatable motor and attached to the main body. Therefore, the robot 100 may acquire information of objects around the robot 100 by rotating the head 360 degrees without controlling the driver 120. However, the embodiment is not limited thereto, and the robot 100 may be implemented as an integrated body that does not distinguish between the head and the main body, in which case the main body and the head may be distinguished on the robot 100 according to the function performed.

[0069] The camera 110 captures objects around the robot 100 and acquires a plurality of images 10 of the objects. Specifically, the camera 110 may acquire a plurality of images 10 of a landing area 3 (e.g., an entrance area 3) and a departure area 4 (e.g., an exit area 4) of a moving passage 200 of the robot 100, and images of a fixed plate 210 and a plurality of movable plates 220 of the moving passage 200. In addition, the camera 110 may acquire images of objects (e.g., humans and / or animals) present in the departure area 4.

[0070] To this end, the camera 110 may be implemented as an imaging device such as a CMOS image sensor (CIS) having a CMOS structure and / or a charge coupled device (CCD) having a CCD structure. However, the camera 110 is not limited thereto, and the camera 110 may be implemented as a camera 110 module capable of capturing various resolutions of an object.

[0071] The camera 110 may be implemented as a depth camera 110, a stereo camera 110, or an RGB camera 110. Therefore, the camera 110 may acquire an image of an object together with depth information of the object.

[0072] The driver 120 is a device capable of enabling the robot 100 to travel. The driver 120 may adjust the travel direction and travel speed under the control of the processor 130. The driver 120 according to the example may include a power generation device (e.g., a gasoline engine, a diesel engine, a liquefied petroleum gas (LPG) engine, an electric motor depending on the fuel (or energy) used) for generating power for the robot 100 to travel, and a steering device (e.g., manual steering, hydraulic steering, and / or electronically controlled power steering (EPS)) for controlling the travel direction.

[0073] The driver 120 operates the plurality of wheels 121 (a first wheel 121, 121-1, a second wheel 121, 121-2, ..., an nth wheel 121, 121-n, where n is a natural number greater than or equal to 2). The plurality of wheels 121 move the robot 100 by rotating based on the control of the processor 130. Specifically, when receiving an electrical signal corresponding to control information (e.g., movement and / or movement speed) generated by the processor 130, the driver 120 may drive the power generation device based on the received electrical signal and rotate the plurality of wheels 121.

[0074] Optionally, based on the control of the processor 130, the robot 100 may rotate by rotating at least one of the plurality of wheels 121 or changing the rotation speed of the plurality of wheels 121. Specifically, when receiving an electrical signal corresponding to the control information (e.g., the rotation direction and / or the rotation speed) generated by the processor 130, the driver 120 may drive the power generation device based on the received electrical signal and rotate at least one of the plurality of wheels 121. Optionally, the driver 120 may receive electrical signals of different amplitudes for each wheel and rotate each wheel at different speeds.

[0075] Hereinafter, the processor 130 generates a control signal related to the rotation and stop of the multiple wheels 121 of the driver 120, sends an electrical signal corresponding to the generated control signal to the driver 120, and the processor 130 controls the rotation or stop of the multiple wheels 121 of the driver 120.

[0076] The driver 120 operates a plurality of suspensions 122 (a first suspension 122-1, a second suspension 122-2, ..., an nth suspension 122-n, where n is a natural number greater than or equal to 2). The suspension 122 is a component for mitigating or absorbing impact generated between the traveling robot 100 and a floor surface.

[0077] The suspension 122 is used to connect each wheel to the robot 100 and the body. Therefore, when the wheel 122 rotates to move the robot 100, the suspension 122 can perform a function of reducing the impact generated between the robot 100 and the floor surface. To this end, the suspension 122 can be implemented as various suspension devices such as an air suspension 122 and a multi-link suspension 122.

[0078] The suspension 122 may be compressed or stretched under the control of the processor 130. When the suspension 122 is compressed, the distance between the wheel 121 connected through the suspension 122 and the main body of the robot 100 decreases, and when the suspension 122 is stretched, the distance between the wheel 121 connected through the suspension 122 and the main body of the robot 100 may increase. Therefore, the robot 100 may compress or stretch the suspension 122 to change the posture of the robot.

[0079] Hereinafter, the processor 130 generates a control signal for stretching or compressing the suspension 122 of the driver 120 , sends an electrical signal corresponding to the generated control signal to the driver 120 , and will be described as the processor 130 controlling to stretch or compress the suspension 122 of the driver 120 .

[0080] Hereinafter, the robot 100 will be described as including four wheels 121 and four suspensions 122 connecting the four wheels 121 to the body of the robot 100. This is for convenience of description, and the robot 100 may be configured to have various numbers of wheels 121 and suspensions 122 according to one or more embodiments.

[0081] The robot 100 may include at least one sensor. The at least one sensor may be implemented as a sensor for detecting objects around the robot 100 (eg, a lidar sensor and / or a ToF sensor) and a sensor for detecting a posture of the robot (eg, an IMU sensor).

[0082] The processor 130 may control general operations of the robot 100. To this end, the processor 130 may include, for example, a random access memory (RAM), a read-only memory (ROM), a central processing unit (CPU), a graphics processing unit (GPU), and / or a system bus, and may perform calculations or data processing related to control of one or more components included in the robot 100.

[0083] The processor 130 may control one or more components included in the robot 100 by executing one or more instructions stored in the memory, controlling one or more components as hardware circuits or chips, or controlling one or more components as a combination of software and hardware.

[0084] The processor 130 may be electrically connected to various components in the robot 100 , including the camera 110 , the driver 120 , at least one sensor (not shown), and a memory (not shown), and control these components.

[0085] When recognizing that the robot 100 is located in the landing area 3 of the moving passage 200 , the processor 130 recognizes the moving direction and moving speed of the moving passage 200 based on the plurality of images 10 of the moving passage 200 acquired through the camera 110 .

[0086] Specifically, the processor 130 recognizes that the robot 100 is located in the boarding area 3 of the moving passage 200 based on the map data of the traveling space in which the robot 100 is located. The map data is map data including information about the traveling space set for the robot 100 to travel. The information about the traveling space includes, for example, a traveling path of the robot 100 set in the traveling space, location information such as a structure (e.g., the moving passage 200), terrain, and / or obstacles in the traveling space. The map data may be stored in a memory of the robot 100 or received from an external server through a communication interface of the robot 100.

[0087] The robot 100 may move in the travel space according to a predetermined travel path based on the map data. In this case, when the travel path is set on the moving passage 200, the processor 130 may control the driver 120 so that the robot 100 moves to the moving passage 200. However, the embodiment is not limited thereto, and the robot 100 may move in the travel space by tracking a user or an external electronic device that communicates with the robot 100. In this case, when the user or the external electronic device moves to the moving passage 200, the processor 130 may control the driver 120 to move the user or the external electronic device to the moving passage 200.

[0088] The processor 130 may recognize that the robot 100 is located in the landing area 3 of the moving passage 200 based on the map data.

[0089] The boarding area 3 refers to an area where a person or robot 100 waits to board the moving passage 200. For example, the boarding area 3 may include a fixed plate 210 installed at the boarding entrance of the moving passage 200. The fixed plate 210 refers to a fixed plate installed at the boarding entrance and exit of the moving passage 200. The fixed plate 210 may also be called a landing plate.

[0090] The fixed plate 210 is distinguished from a plurality of moving plates 220 constituting a conveyor belt of the moving channel 200. The moving plate 220 is disposed at the upper end of the rollers of the moving channel 200 and is moved in a specific direction by a driver and a transmission chain. The plate may also be referred to as, for example, a step, a footrest, and / or a pedal.

[0091] The exit area 3 may further include a floor surface surrounding the exit of the moving passage 200. For example, the boarding area 3 may include a fixed plate 210 installed at the boarding entrance of the moving passage 200 and a floor surface of a predetermined range around the fixed plate 210.

[0092] Such a boarding area 3 may be pre-set in the map data. Therefore, the processor 130 may recognize that the robot 100 is located in the boarding area 3 based on the map data.

[0093] The processor 130 may recognize that the robot 100 is located in the boarding area 3 by recognizing a column beam mounted on the fixing plate 210 of the boarding area 3. For example, when the robot 100 recognizes a column beam on an image acquired by the camera 110 while traveling, the processor 130 may recognize that the robot 100 is located in the boarding area 3. To this end, the processor 130 may use a neural network model trained to recognize a column beam.

[0094] The neural network model may be a convolutional neural network (CNN) including a convolutional layer for extracting image feature information and a fully connected layer trained to identify objects included in the image based on the extracted feature information. The neural network model may be stored in a memory. In addition, the processor 130 may communicate with the column beam of the boarding area 3 through the communication interface of the robot 100 to identify the distance between the robot 100 and the column beam. In this case, when the identified distance is within a predetermined distance, the processor 130 may recognize that the robot 100 is located in the boarding area 3.

[0095] In addition, when the moving plate 220 is recognized in the image acquired through the camera 110 when the robot 100 travels, the processor 130 may recognize that the robot 100 is located in the boarding area 3. For example, the processor 130 may recognize the moving plate 220 of the moving passage 200 in the image acquired through the camera 110 when the robot 100 travels. Specifically, when the robot 100 moves toward the moving passage 200, the processor 130 may repeatedly acquire an image of the moving passage 200 through the camera 110. The processor 130 may recognize that the moving area of ​​the moving passage 200 is gradually expanded on the image acquired through the camera 110.

[0096] The moving area may be an area of ​​the moving plate 220 including the moving passage 200. Therefore, when the robot 100 approaches the moving plate 200, the processor 130 may recognize that the moving area is gradually expanded on the image acquired in real time through the camera 110. When the moving area is greater than or equal to a predetermined range, the processor 130 may recognize that the robot 100 is located in the boarding area 3.

[0097] In addition, the robot 100 can identify whether the robot 100 is located in the boarding area 3 by a neural network model trained to identify the boarding area 3 on the image. Specifically, the processor 130 can identify whether the robot is located in the boarding area 3 by using a neural network model trained by a plurality of images (i.e., training data), wherein the plurality of images are obtained in various boarding areas 3 including the boarding area 3 of the moving passage 200 in the travel space where the robot is located. The neural network model for detecting the boarding area 3 included in the image can be implemented as, for example, a convolutional neural network (CNN) model, a fully convolutional network (FCN) model, a region having convolutional neural network features (RCNN) model, and / or a YOLO model. The neural network model can be stored in a memory.

[0098] In this manner, the processor 130 may recognize that the robot 100 is located in the boarding area 3 through various methods.

[0099] When recognizing that the robot 100 is located in the boarding area 3, the processor 130 obtains a plurality of images 10 of the moving passage 200 through the camera 110. The processor 130 recognizes the moving direction and moving speed of the moving passage 200 based on the obtained plurality of images 10 of the moving passage.

[0100] Specifically, when it is recognized that the robot 100 is located in the boarding area 3, the processor 130 may control the driver 120 to stop the robot 100. When the robot 100 stops, the processor 130 may obtain a plurality of images 10 of the moving channel 200 through the camera 110.

[0101] In this case, the processor 130 may control the driver 120 to move the robot 100 to acquire a plurality of images 10. Specifically, when it is detected that the robot 100 is located in the boarding area 3, the processor 130 may control the driver 120 to move the robot 100 to an image acquisition point in the boarding area 3.

[0102] The image acquisition point may be a position set to acquire an image of the moving passage 200 without interfering with a user or another robot 100 boarding the moving passage 200. The image acquisition point may be set in advance for each boarding area on the map data.

[0103] Optionally, when it is recognized that the robot 100 is located in the boarding area, the processor 130 may acquire at least one image of the moving passage 200 through the camera 110, and recognize the width of the moving passage 200 through the acquired at least one image. The width of the moving passage 200 may be recognized based on the detection of the moving plate 220 in the acquired at least one image and the detected width of the moving plate 220. The processor 130 may select an image acquisition point of the robot 100 based on the recognized width of the moving passage. As an example, based on the recognized width of the moving passage 200 and the size information about the body of the robot 100 (e.g., the length information about the body of the robot 100), the driver 120 may be controlled so that the robot 100 is located on one side of the boarding entrance of the moving passage 200.

[0104] The processor 130 may identify the moving direction and moving speed of the moving passage 200 through the acquired multiple images 10 of the moving passage 200. Specifically, the processor 130 may detect the moving plate 220 within the acquired multiple images 10 of the moving passage 200. The processor 130 may identify the position of the moving plate 220 detected in each image, and identify the change in the position of the identified moving plate 220. The processor 130 may identify the moving direction of the moving passage 200 based on the change in the position of the identified moving plate 220, and may also identify the moving speed of the moving passage 200 based on the change in the position of the identified moving plate 220 and the frame rate of the multiple images 10.

[0105] Hereinafter, an embodiment for identifying a moving direction and a moving speed of the moving passage 200 will be described in detail.

[0106] Figure 3 is an exemplary diagram illustrating a method of identifying a moving direction and a moving speed of a moving channel 200 according to one or more embodiments of the present disclosure.

[0107] Reference Figure 3 When it is recognized that the robot 100 is located in the boarding area 3, the processor 130 may obtain a plurality of images 11 and 12 of the moving passage 200 in the boarding area 3 through the camera 110. Specifically, the processor 130 may obtain a first image 11 of the moving passage 200 at time t1 and obtain a second image 12 of the moving passage 200 at time t2.

[0108] The processor 130 may set a region of interest (ROI) 20 of the plurality of images 10 acquired by the camera 110. Specifically, the processor 130 may identify a variable region within the image by comparing the acquired plurality of images 10. Since the fixed plate within the plurality of images 10 remains fixed and does not move, the variable region may be identified based on the movement of the moving plate 220 detected within the plurality of images 10. The processor 130 may set a region corresponding to the variable region as the region of interest.

[0109] The processor 130 may identify a change in the position of at least one moving plate 220 within the set region of interest.

[0110] Reference Figure 3 , the processor 130 may recognize the change in position of the first moving board 221 and the second moving board 222 in the first image 11 and the second image 12. The processor 130 may recognize the moving direction and the moving speed of the moving passage 200 based on the change in position of each moving board 220 (the first moving board 221 and the second moving board 222) in the first image 11 and the second image 12, respectively. In this case, the processor 130 may recognize that the moving direction of the moving passage 200 is north.

[0111] The processor 130 may set an identifier (eg, a board ID) for each mobile board 220 detected in the plurality of images 10 , and track each mobile board 220 .

[0112] When the robot 100 is located in the boarding area 3, the processor 130 can identify the boundary lines between the multiple plates of the mobile channel 200 based on the multiple images 10 acquired by the camera 110, identify the moving direction of the mobile channel 200 based on the normal vector 40 of the identified boundary lines, and identify the moving speed of the mobile channel 200 based on the moving speed of the identified boundary lines.

[0113] Figure 4 is an exemplary diagram of a method of identifying a moving direction and a moving speed of a moving channel based on a boundary line between plates according to one or more embodiments of the present disclosure.

[0114] Specifically, when the robot 100 is located in the boarding area 3 , the processor 130 may acquire a plurality of images 10 of the moving channel 200 through the camera 110 , and identify boundary lines between a plurality of adjacent moving plates 220 of the moving channel 200 based on the acquired plurality of images 10 .

[0115] The processor 130 may extract linear components within each image by applying a linear detection algorithm to each of the plurality of images 10. For example, the processor 130 may extract linear components included in each image by Hough transformation. The extracted linear components may include a boundary between the fixed plate 210 and the mobile plate 220, a boundary between a plurality of adjacent mobile plates 220, a safety line marked on the mobile plate 220, and a handrail of the mobile passage 200.

[0116] In this manner, the processor 130 may identify, among the linear components extracted from each image, a boundary line corresponding to the boundary between the plurality of adjacent moving plates 220. Specifically, the processor 130 may identify the linear component within the ROI 20 set in each image as the boundary line between the moving plates 220. Alternatively, the processor 130 may identify feature information of each linear component, and select the boundary line between the plurality of moving plates 220 from among the plurality of linear components based on the identified feature information. Alternatively, the processor 130 may compare the plurality of images 10 to detect a linear component whose position changes in each image, and identify the detected linear component as the boundary line between the plurality of moving plates 220.

[0117] The processor 130 may track the boundary lines between the plurality of adjacent moving plates 220 identified in the plurality of images 10 and identify the moving direction and moving speed of the boundary lines.

[0118] Specifically, the processor 130 may track the identified boundary line to identify a position change of the boundary line in each image, and identify a moving speed of the boundary line based on the position change of the identified boundary line and a frame rate of each image.

[0119] Reference Figure 4 , the processor 130 may respectively detect linear components included in the ROI 20 set in the third image 13 acquired at time t3 and the fourth image 14 acquired at time t4.

[0120] Specifically, the plurality of linear components detected in the third image 13 and the fourth image 14 may include a boundary line between the fixed plate 210 and the moving plate 220, a boundary line between the handrail table and the moving plate 200, and a boundary line between a plurality of adjacent moving plates 220. In this case, the processor 130 may identify the boundary lines between the plurality of moving plates 220 included in the ROI and track the identified boundary lines.

[0121] The processor 130 may extract the normal vector 40 of the boundary line between the plurality of adjacent moving plates 220, and identify the moving direction of the moving channel 200 based on the extracted normal vector 40. Specifically, the processor 130 may identify the normal vector 40 perpendicular to the boundary line between the moving plates 220. The processor 130 may identify the direction of the identified normal vector 40 as the direction in which the position of the moving plate 220 is changed. For example, the processor 130 may identify the coordinate value of the boundary line in a two-dimensional (2D) coordinate space based on the position of the boundary line between the moving plates 220 in the image, and identify the normal vector 40 of the boundary line in the 2D coordinate space based on the identified coordinate value.

[0122] Return to reference Figure 4 , the processor 130 may respectively identify the boundary lines 31 of the third moving plate 221 and the fourth moving plate 222 in the third image 13 acquired at time t3 and the fourth image 14 acquired at time t4, and may extract the normal vector 40 of the boundary line 31 in the fourth image. This may be identified based on the position change of the boundary line 31 in the third image and the boundary 31 in the fourth image. The processor 130 may identify the direction of the extracted normal vector 40 as the moving direction of the moving channel 200.

[0123] The processor 130 may select at least one plate from among the plurality of moving plates 220 detected in the plurality of images 10 acquired through the camera 110 for identifying the moving direction and the moving speed of the moving passage 200 .

[0124] Specifically, the processor 130 may acquire distance information about the plurality of moving plates 220 detected in the plurality of images 10. The distance information about the moving plates 220 may be acquired by the camera 110 (e.g., the stereo camera 110 and / or the RGB camera 110) and / or the sensor (the lidar sensor and / or the ToF sensor). The processor 130 may acquire 3D position information (e.g., coordinate values ​​in a predetermined 3D coordinate space) of the plurality of moving plates 220 based on the acquired distance information. The processor 130 may detect the moving plate 220 having the lowest height (e.g., z-axis position) among the plurality of moving plates 220 based on the 3D position information of the plurality of moving plates 220. The processor 130 may identify the moving direction and moving speed of the moving channel 200 based on the detected position change of the moving plate 220.

[0125] The processor 130 identifies the moving direction and moving speed of the moving passage 200, and then controls the driver 120 so that the robot 100 faces a mounting direction (e.g., an entry direction) corresponding to the identified moving direction, and controls the suspension 122 connected to at least one of the plurality of wheels 121 so that the robot 100 tilts at an angle corresponding to the identified moving speed. The processor 130 may control the driver 120 so that the tilted robot 100 moves in the mounting direction and mounts the moving passage 200.

[0126] Figure 5 is an example diagram showing how to determine the boarding direction of the robot based on the moving direction of the moving channel according to one or more embodiments of the present disclosure, Figure 6 is an example diagram illustrating determining a tilt angle of a robot based on a moving speed of a moving channel according to one or more embodiments of the present disclosure.

[0127] Specifically, the processor 130 may identify the boarding direction of the robot 100 based on the identified moving direction of the moving channel 200. For example, the processor 130 may identify the direction of the normal vector 40 of the boundary line between the moving plates 220 of the moving channel 200 as the boarding direction of the robot 100. That is, the processor 130 may identify the moving direction of the moving channel 200 as the boarding direction of the robot 100.

[0128] The processor 130 may control the driver 120 so that the robot 100 faces the identified boarding direction. That is, the processor 130 may control the driver 120 of the robot 100 so that the vector facing the body of the robot 100 matches the extracted normal vector 40. To this end, the processor 130 may generate control information based on the moving direction of the moving channel 200, and rotate the wheels 121 included in the driver 120 based on the generated control information to control the robot 100 to face the boarding direction. That is, the processor 130 may control the driver 120 to align the robot 100 in the moving direction of the moving channel 200.

[0129] Reference Figure 5 , the processor 130 recognizes the moving direction of the moving passage 200 as north. In this case, the processor 130 may recognize the boarding direction of the moving passage 200 of the robot 100 as north based on the moving direction of the moving passage 200. The processor 130 may rotate the wheels so that the robot 100 faces north as the boarding direction. Therefore, the robot 100 facing the northeast direction may rotate in the north direction matching the moving direction of the moving passage 200.

[0130] The processor 130 may control the driver 120 so that the robot 100 faces the boarding direction, and then control the suspension 122 connected to at least one of the plurality of wheels 121 so that the robot 100 tilts at an angle corresponding to the identified moving speed of the moving channel 200 .

[0131] Specifically, when the robot 100 is aligned toward the boarding direction, the processor 130 may control the driver 120 to identify an angle corresponding to the moving speed of the moving channel 200. The processor 130 may identify the angle corresponding to the moving speed of the moving channel 200 as the pitch angle of the robot 100. In this case, the angle corresponding to the moving speed of the moving channel 200 may be identified in consideration of, for example, the moving speed of the moving channel 200, the friction coefficient of the moving plate 220, the weight of the robot 100, and / or the weight of an object carried by the robot 100.

[0132] For example, the angle corresponding to the moving speed of the moving passage 200 may be identified based on Equation 1 below.

[0133] Equation 1

[0134]

[0135] θ represents an angle corresponding to the moving speed of the moving channel 200 , v represents the moving speed of the moving channel 200 , m represents the mass of the robot 100 , g represents the gravitational acceleration, and c represents a coefficient set in consideration of, for example, the friction coefficient of the moving plate 220 .

[0136] The processor 130 may control the suspension 122 connected to at least one of the plurality of wheels 121 so that the robot 100 is tilted at an angle corresponding to the moving speed of the moving channel 200 identified based on the moving speed of the moving channel 200. Specifically, the processor 130 may identify the length of the suspension 122 of each of the plurality of wheels 121 that causes the robot 100 to tilt at an angle corresponding to the moving speed of the moving channel 200, and may identify at least one wheel whose length of the suspension 122 should be changed from among the plurality of wheels 121. The processor 130 may generate control information such that the length of the suspension 122 of the identified at least one wheel is adjusted, and may control the driver 120 based on the generated control information so that the robot 100 is tilted at an angle corresponding to the moving speed of the moving channel 200.

[0137] For example, the processor 130 may control the suspension 122 connected to at least one of the multiple wheels 121 to be stretched, or the processor 130 may control the suspension 122 connected to at least one of the multiple wheels 121 to be compressed, so that the robot 100 can tilt at an angle corresponding to the moving speed of the identified moving channel 200.

[0138] Reference Figure 6 , the processor 130 may respectively control the suspensions 122 connected to the third wheel 121 and the fourth wheel 121 located behind the plurality of wheels 121 so that the robot 100 moves at an angle θ1 corresponding to the moving speed v1 of the identified moving channel 200. Specifically, Figure 6 In the embodiment, the processor 130 stretches the suspension 122 connected to the third wheel 121 and the suspension 122 connected to the fourth wheel 121 so that the robot 100 tilts.

[0139] The processor 130 controls the driver 120 so that after the robot 100 is tilted at an angle corresponding to the moving speed of the moving passage 200 , the tilted robot moves in a mounting direction and mounts the moving passage 200 .

[0140] Specifically, when the robot 100 is tilted at an angle corresponding to the moving speed of the moving channel 200, the processor 130 may control the driver 120 to move toward the mounting direction. In this case, the processor 130 may determine the moving speed (i.e., the mounting speed) at which the robot 100 mounts the moving channel 200 based on the moving speed of the moving channel 200. For example, the processor 130 may determine the mounting speed of the moving channel 200 as a speed increased by a predetermined ratio from the identified speed of the moving channel 200.

[0141] As in one embodiment described above, the processor 130 may identify the moving direction and moving speed of the moving channel 200 that the robot 100 intends to board based on the image acquired by the camera 110 in the boarding area 3. The posture of the robot and the moving direction of the robot 100 suitable for the identified moving direction and moving speed are determined. Specifically, although the moving direction and moving speed of each moving channel 200 may be different, the robot 100 may distinguish and identify the moving direction and moving speed of each moving channel 200. In addition, after the robot boards the moving channel 200, the posture of the robot is not determined in consideration of the moving direction and moving speed of the moving channel 200, but before the robot 100 boards the moving channel 200, since the boarding posture of the robot 100 is determined in consideration of the moving direction and moving speed of the moving channel 200 in advance, it is possible for the robot 100 to smoothly board the moving channel 200.

[0142] When the robot 100 is located in the boarding area 3, the processor 130 can identify the width of the moving channel 200 based on the multiple images 10 acquired by the camera 110, determine the boarding position (e.g., the entrance position) of the moving channel 200 of the robot 100 based on the identified width of the moving channel 200, and control the driver 120 to face the boarding direction corresponding to the identified moving direction of the robot 100 at the determined boarding position.

[0143] Specifically, when the robot 100 is located in the boarding area 3, the processor 130 may identify the width of the moving passage 200 based on the plurality of images 10 acquired by the camera 110. The processor 130 may identify the width of the moving passage 200 based on the length information of the boundary line between the moving plates detected in the image. Alternatively, the processor 130 may identify the width of the moving passage 200 based on the width of the ROI set in the image.

[0144] In this way, the processor 130 may determine the boarding position of the moving passage 200 of the robot 100 based on the identified width of the moving passage 200. For example, the processor 130 may determine the boarding position of the robot 100 as the center of the moving passage 200 based on the width of the moving passage 200. The processor 130 may control the driver 120 so that the robot 100 is located at the determined boarding position. In addition, the processor 130 may control the driver 120 so that the robot 100 faces the boarding direction corresponding to the identified moving direction of the robot 100 at the determined boarding position. The processor 130 may control the suspension 122 connected to at least one wheel so that the robot 100 tilts at an angle corresponding to the moving speed of the moving passage 200, and control the driver 120 so that the tilted robot 100 moves from the determined boarding position in the boarding direction and boards the moving passage 200.

[0145] In the following, reference is made to Figures 7 to 9 , a detailed method of aligning the posture of the robot based on the inclination angle of the moving channel 200 after the robot 100 climbs onto the moving channel 200 will be described.

[0146] When recognizing that the robot 100 has boarded the moving passage 200 , the processor 130 may recognize a plurality of boards of the moving passage 200 based on a plurality of images 10 of the moving passage 200 acquired through the camera 110 .

[0147] Specifically, the processor 130 may recognize whether the robot 100 has boarded the moving passage 200. In this case, when it is recognized that the position of the robot 100 is changed without the control of the driver 120 for the map data, the processor 130 may recognize that the robot 100 has boarded the moving passage 200. Alternatively, when the processor 130 detects that the plurality of wheels 121 of the driver 120 are located on the moving passage 200, the processor 130 may recognize that the robot 100 has boarded the moving passage 200.

[0148] When recognizing that the robot 100 has boarded the moving passage 200 , the processor 130 may control the driver 120 to stop the robot 100 , and when the robot 100 stops, acquire a plurality of images 10 of the moving passage 200 through the camera 110 .

[0149] In this case, the processor 130 may control the suspension 122 of at least one of the plurality of wheels 121 so that the tilted robot 100 stands upright before acquiring the plurality of images 10 or while acquiring the plurality of images 10 .

[0150] Specifically, the processor 130 may restore the tilted robot 100 to its original state based on the moving speed of the moving channel. The original state of the robot 100 may be a state of the robot 100 before mounting the moving channel 200 or a state when the robot 100 is standing upright. The state of the robot 100 when standing upright may be a state in which the robot 100 is parallel to the floor surface on which the moving channel 200 is installed. For example, the processor 130 may control the suspension 122 connected to at least one wheel that is stretched to be compressed based on the moving speed of the moving channel 200, or control the suspension 122 connected to at least one wheel that is compressed to be stretched based on the moving speed of the moving channel 200, so that the robot 100 may be parallel to the floor surface on which the moving channel 200 is installed.

[0151] In addition, when it is recognized that the processor 130 has boarded the moving channel 200, the processor 130 may align the robot 100 based on the recognized boarding direction. For example, when the robot 100 boards the moving channel 200, the direction of the robot 100 matching the moving direction of the moving channel 200 may be distorted when boarding, for example, by friction generated between the moving channel 200 (e.g., the moving plate 220 of the moving channel 200) and the robot 100 (e.g., the wheels 121 of the robot 100).

[0152] Therefore, when it is recognized that the robot 100 has boarded the moving passage 200, the processor 130 may recognize whether the robot is aligned in the moving direction of the moving passage 200 immediately after the robot 100 boards the moving passage 200 based on the boarding direction recognized before the boarding. When it is recognized that the robot 100 is not aligned in the moving direction of the moving passage 200, the processor 130 may control the wheels 121 of the driver 120 such that the robot 100 is aligned in the moving direction of the moving passage 200 based on the boarding direction recognized before the boarding.

[0153] The processor 130 may identify a plurality of plates of the moving passage 200 in the acquired plurality of images 10. The plurality of plates identified in the plurality of images 10 acquired when the robot 100 is stationary on the moving passage 200 may be the moving plates of the moving passage 200 described above.

[0154] However, unlike the moving plate identified in the plurality of images 10 acquired in the boarding area 3, the change in the position of the moving plate 220 may not be recognized in the plurality of images 10 acquired when the robot 100 boards the moving path 200. This is because the robot 100 and the moving plate 220 move at the same moving speed when the robot 100 boards the moving path 200. That is, the relative speed between the moving plate 220 and the robot 100 on the moving path 200 is a value of "0".

[0155] The processor 130 may identify the plurality of moving plates 220 of the moving passage 200 through the plurality of images 10 acquired through the camera 110. In this case, the plurality of moving plates 220 identified by the processor 130 through the plurality of images 10 may include the plurality of moving plates 220 located in front of the robot 100 and the moving plates below the robot 100 on which the robot 100 is standing.

[0156] The processor 130 may identify the distance between the robot 100 and each of the moving plates 220 through the acquired plurality of images 10. For example, the processor 130 may respectively identify the distances between the plurality of moving plates 220 and the robot 100 through the plurality of images 10 acquired by the camera 110 implemented as, for example, the depth camera 110 and / or the stereo camera 110. The processor 130 may identify the position of each of the moving plates 220 relative to the robot 100 based on the identified distance between each of the moving plates 220. For example, the processor 130 may identify the identified position of each of the moving plates 220 in the 3D coordinate space.

[0157] The processor 130 may identify the plurality of mobile boards 220 using a sensor (e.g., a laser radar sensor or a ToF sensor) of the robot 100. For example, the processor 130 may identify a position where a laser emitted from the laser radar sensor is reflected after reaching an object around the robot 100 as one point. The processor 130 may identify a group of cluster points corresponding to each mobile board 220 among the identified plurality of points. In this case, the processor 130 may identify a group of cluster points corresponding to each mobile board 220 based on, for example, a random sample consensus algorithm (RANSAC) technique and / or a principal component analysis (PCA) technique.

[0158] In this manner, the processor 130 may identify a plurality of mobile plates 220 around the robot 100 , and identify the distance of each mobile plate 220 from the robot 100 and the position of each plate within the 3D coordinate space 300 .

[0159] The processor 130 may recognize the plurality of moving boards 220 by combining the plurality of images 10 acquired via the camera 110 with sensing information (point cloud regarding the board) acquired via the sensor.

[0160] In addition, the processor 130 may recognize a plurality of boards within the plurality of images 10 acquired via the camera 110 through various methods, and recognize the position of each of the recognized plurality of boards.

[0161] Figure 7 2 is an exemplary diagram showing that the robot 100 boarding the moving passage 200 recognizes a plurality of boards according to one or more embodiments of the present disclosure.

[0162] Reference Figure 7 , the processor 130 may detect a plurality of moving plates 220 (i.e., the fifth moving plate 225, the sixth moving plate 226, and the seventh moving plate 227) on the image acquired by the camera 110. The processor 130 may recognize the position of each plate in the predetermined 3D coordinate space 300 based on the distance information of each plate acquired by the camera 110 or the sensor.

[0163] In addition, as in the above-mentioned landing area, the processor 130 may apply the Hough transform to each image so as to detect a plurality of mobile boards 220 on each image, or set identifiers for the detected plurality of mobile boards 220. In this regard, the contents described with reference to the operation of the processor 130 in the landing area may be similarly applied.

[0164] After identifying the plurality of moving plates 220 of the moving passage 200 , the processor 130 may identify the tilt angle of the moving passage 200 based on the normal vector 40 of at least one of the identified plurality of moving plates 200 .

[0165] Specifically, the processor 130 may respectively extract the normal vectors 40 of the identified plurality of mobile boards 220. For example, the processor 130 may identify the surfaces 225', 226', and 227' corresponding to each mobile board 220 based on the position information of each identified mobile board 220 in the 3D coordinate space 300. In this case, the coordinates of the surfaces 225', 226', and 227' corresponding to each mobile board may be identified by, for example, a matrix, a vector.

[0166] The processor 130 may extract the normal vector 40 of the surface 225', 226', and 227' corresponding to each mobile plate. In this case, each normal vector 40 of the surface 225', 226', and 227' corresponding to each mobile plate may be extracted in the same z-axis direction (e.g., +z-axis direction). In this case, for example, the processor 130 may extract the normal vector of the surface 225', 226', and 227' corresponding to each mobile plate 220 based on, for example, the least square method. Return to reference Figure 7The processor 130 may extract a normal vector 41 of a surface 225′ corresponding to the fifth moving plate 225, a normal vector 42 of a surface 226′ corresponding to the sixth moving plate 226, and a normal vector 43 of a surface 227′ corresponding to the seventh moving plate 227 in the 3D coordinate space based on the position information of each moving plate.

[0167] After extracting the normal vectors 40 of the moving plate 220 in this manner, the processor 130 may recognize the tilt angle of the moving passage 200 based on each normal vector.

[0168] Specifically, the processor 130 may identify the position and direction of each normal vector 40 based on the posture of the robot, and identify the inclination angle formed by each moving plate (specifically, the inclination angle between each moving plate and the floor surface on which the moving channel 200 is installed).

[0169] For example, when the processor 130 controls the driver 120 to make the robot 100 stand (or be parallel to the floor surface on which the moving channel 200 is installed) after the robot 100 steps onto the moving channel 200, the processor 130 can identify the angle between the normal vector of the robot 100 and each moving plate in the 3D coordinate space 300, and identify the tilt angle formed by each moving plate based on the identified angle.

[0170] Hereinafter, for convenience of description, it is assumed that the x component of the normal vector identified in the 3D coordinate space 300 is 0.

[0171] Figure 8 2 is an exemplary diagram of a method of identifying the tilt angle of the moving channel 200 based on the normal vector of the board according to one or more embodiments of the present disclosure.

[0172] Reference Figure 8 , the processor 130 may extract the normal vector 41 of the fifth moving plate 225 and identify the position and direction of the normal vector relative to the robot 100 in the 3D coordinate space 300. Specifically, the processor may identify θ1 as the tilt angle of the fifth moving plate 225 based on the vector in the direction facing the robot body (i.e., the vector of the y component), the vector in the vertical direction of the head and the body of the robot (i.e., the z component vector), and the normal vector 41 corresponding to the fifth moving plate 225. The processor 130 may extract the normal vector 42 corresponding to the sixth moving plate 226 and identify the position and direction of the normal vector 42 relative to the robot 100 in the 3D coordinate space 300. Based on the position and direction of the normal vector, θ2 may be identified, where θ2 is the tilt angle of the sixth moving plate 226. Similarly, the processor 130 may identify θ3 based on the normal vector 43 of the seventh moving plate 227, where θ3 is the tilt angle of the seventh moving plate 227.

[0173] Fig. 9 is an example diagram of a method of controlling a suspension of at least one wheel of a robot based on an identified tilt angle according to one or more embodiments of the present disclosure.

[0174] The processor 130 may control the suspension 122 of at least one wheel based on the identified tilt angle so that the robot 100 stands upright. That is, the suspension 122 of at least one wheel may be controlled so that the robot 100 is parallel to the floor surface on which the moving channel 200 is installed.

[0175] Specifically, the processor 130 may identify the rotation angle (roll angle and pitch angle) of the robot 100 based on the identified tilt angle. The processor 130 may stretch or compress the suspension 122 of at least one wheel so that the robot 100 rotates based on the identified rotation angle of the robot 100.

[0176] The processor 130 may respectively identify the tilt angles of the plurality of moving plates 220 located in front of the robot 100 , and then identify a moment when the moving plate 220 on which the robot 100 is standing moves to the position of the moving plate 220 located in front.

[0177] For example, refer to Fig. 9 , the eighth moving plate 228 on which the robot 100 is standing moves to the point where the fifth moving plate 225 is located at time t1 at time t2. In this case, the processor 130 may predict the inclination angle of the eighth moving plate on which the robot 100 is located at time t2 based on the inclination angle of the fifth moving plate 225 recognized from the image acquired at time t1.

[0178] Therefore, when the eighth moving plate 228 on which the robot 100 is standing at time t1 passes d1 and moves to d2 (i.e., the position of the fifth moving plate at time t1), the processor 130 may control the suspension 122 of at least one wheel based on the identified tilt angle so that the robot 100 is parallel to the floor surface on which the moving channel 200 is installed. Fig. 9 , based on the identified inclination angle θ1, the processor 130 may respectively stretch the suspensions (i.e., the third suspension 122-3 and the fourth suspension 122-4) connected to the third wheel 121-3 and the fourth wheel 121-4 of the robot 100. Specifically, the processor 130 may generate control information about each of the third suspension 122-3 and the fourth suspension 122-4 so that the length of the third suspension 122-3 and the fourth suspension 122-4 increases by l2, and stretch each of the third suspension 122-3 and the fourth suspension 122-4 based on the generated control information.

[0179] Therefore, at time t2, the robot 100 may be parallel to the floor surface on which the moving channel 200 is installed. Fig. 9, it is shown that only the pitch angle of the robot 100 is adjusted, but this is just for convenience of description, and the processor 130 may adjust the yaw angle as well as the pitch angle of the robot 100 based on the tilt angle of each moving plate 220 .

[0180] The processor 130 controls the suspension 122 of at least one wheel based on the identified tilt angle so that the time for the robot 100 to be parallel to the floor surface on which the moving channel 200 is installed can be set based on the moving speed of the moving channel 200 and the length information of the moving plate 220. That is, referring again to Fig. 9 , the processor 130 may determine that the eighth moving plate 228 where the robot 100 is located at time t1 is located at d2 at time t2 (i.e., the position of the fifth moving plate 225 at time t1) based on the moving speed v of the moving channel 200 and the length information l of the moving plate 220. Therefore, the processor 130 may control the suspension 122 of at least one wheel based on the tilt angle of the fifth moving plate 225 identified at time t1 before time t2 arrives (e.g., before a predetermined time from time t2). Therefore, even if the eighth moving plate 228 where the robot 100 is standing moves at d2, the robot 100 can be parallel to the floor surface as at d1.

[0181] Although not clearly shown in the drawings, the processor 130 may determine that the eighth moving plate 228 where the robot 100 is located at a time after time t2 (e.g., time t3) is located at d3 (i.e., the position of the sixth moving plate at time t1) based on the moving speed v of the moving channel 200 and the length information l of the moving plate 220. The length information l of the moving plate 220 may be identified based on, for example, distance information about the moving plate 220 acquired by the camera 110 and / or a point cloud of the moving plate 220 acquired by a sensor.

[0182] In this case, the processor 130 may control the suspension 122 of at least one wheel of the robot 100 based on the inclination angle of the sixth moving plate identified at time t1. For example, the processor 130 may identify the difference between the inclination angle of the fifth moving plate 225 and the inclination angle of the sixth moving plate 226 identified at time t1 based on the normal vector of the fifth moving plate 225 and the normal vector of the sixth moving plate. The processor 130 may generate control information for the suspension 122 connected to at least one wheel based on the identified difference in inclination angles so that the robot 100 is parallel to the floor surface on which the moving channel 200 is installed at time t3 based on the posture of the robot at time t2. The processor 130 may adjust the posture of the robot 100 based on the generated control information (e.g., stretched or compressed length information of the suspension 122 connected to at least one wheel) so that the robot 100 is parallel to the floor surface at time t3.

[0183] However, the embodiments are not limited thereto, and the processor 130 may re-extract the normal vector of the fifth mobile plate 225 located at d3 through the multiple images 10 acquired at time t2, and may identify the position and direction of the normal vector of the fifth mobile plate 225 based on the posture of the robot in the 3D coordinate space 300 at time t2 (i.e., the horizontal posture of the robot 100 relative to the floor surface).

[0184] The processor 130 may identify the inclination angle of the fifth moving plate 225 at time t2. In this case, the processor 130 may identify the relative angle difference between the eighth moving plate 228 and the fifth moving plate 225 where the robot 100 is located at time t2. The processor 130 may control the suspension 122 connected to at least one wheel based on the identified relative angle difference or the inclination angle of the moving plate 225 so that the robot 100 is parallel to the floor surface on which the moving channel 200 is installed.

[0185] In this way, even after the suspension 122 of at least one wheel is controlled based on the recognized tilt angle so that the robot 100 is parallel to the floor surface on which the moving passage 200 is installed, the processor 130 can recognize the plurality of moving plates 220 of the moving passage 200 again based on the image acquired through the camera 110, and continuously recognize the tilt angle of the moving passage 200 based on the normal vector of at least one of the recognized plurality of moving plates 220. Therefore, even if the tilt angle of the moving plate 220 on which the robot 100 is located changes, the processor 130 can predict the tilt angle of the moving plate 220 in advance and control the posture of the robot to be flush with the floor surface.

[0186] When the robot 100 gets on the moving passage 200, the processor 130 may control the suspension 122 connected to at least one wheel based on the recognized tilt angle, and may rotate at least one wheel to adjust the direction of the robot 100. Specifically, the processor 130 may recognize boundary lines between the plurality of moving plates 220 based on an image of the moving passage 200 acquired through the camera 110.

[0187] That is, similar to what is done in the boarding area 3, the processor 130 may detect the boundary lines between the plurality of moving plates 220 of the moving channel 200 through a straight line detection algorithm (e.g., Hough transform). The processor 130 may extract the normal vector of the detected boundary line and continuously identify the moving direction of the moving channel 200 based on the extracted normal vector. The processor 130 may control the driver 120 to rotate at least one wheel so that the posture of the robot faces the moving direction of the moving channel 200.

[0188] The processor 130 may select at least one of the plurality of moving plates 220 detected on the image acquired by the camera 110 based on the moving speed of the moving channel 200. The processor 130 may extract only the normal vector of the selected at least one plate. Hereinafter, the selected at least one plate will be referred to as a target plate.

[0189] Reference Fig. 9 , the processor 130 may select only the fifth moving plate 225 and the sixth moving plate 226 as the target plate among the plurality of moving plates 220 (the fifth moving plate 225 to the seventh moving plate 227), and extract only the normal vectors of the fifth moving plate 225 and the sixth moving plate 226. The reason is that even if the normal vector of the seventh moving plate 227 is extracted to identify the tilt angle of the seventh moving plate 227 at time t1, the tilt angle of the moving plate 220 at position d3 (i.e., the seventh moving plate at time t1) may be identified by the normal vector of the sixth moving plate 226 identified at time t2. Therefore, the processor 130 may select the moving plate (i.e., the target plate) from which the normal vector 40 is extracted from the plurality of moving plates 220 detected through the plurality of images 10, thereby preventing unnecessary waste of resources of the robot 100.

[0190] The target plate may be selected based on the moving speed of the moving passage 200. That is, when the moving speed of the moving passage 200 is slow, a small number of target plates may be selected, and when the moving speed of the moving passage 200 is fast, a large number of target plates may be selected. As the moving speed of the moving passage 200 increases, the tilt angle of the moving plate 220 on which the robot 100 stands also changes rapidly. Therefore, the reason why the processor 130 predicts the tilt angle of the moving plate 220 away from the robot 100 is that the robot 100 can take a more stable posture in the moving passage 200.

[0191] In addition, the processor 130 may identify at least one of the plurality of panels based on the moving speed of the moving channel 200 and the response speed of the suspension 122 .

[0192] Specifically, the processor 130 may select a target plate from among a plurality of moving plates detected on a plurality of images 10 acquired by the camera 110 based on the moving speed of the moving channel 200 and the response speed of the suspension 122. Fig. 9 When the response speed of the suspension 122 is slower than the moving speed of the moving channel 200, the processor 130 may set the sixth moving plate 226 among the plurality of moving plates 220 (the fifth moving plate 225 to the seventh moving plate 225227) identified at time t1 as the target plate. Alternatively, when the response speed of the suspension 122 is faster than the moving speed of the moving channel 200, the processor 130 may set all of the plurality of moving plates 220 (the fifth moving plate 225 to the seventh moving plate 227) identified at time t1 as the target plate.

[0193] In this way, the processor 130 can compare the identified moving speed of the moving channel 200 and the response speed of the suspension 122, and flexibly set the target plate based on the comparison result. Therefore, the processor 130 can appropriately change the posture of the robot according to the moving speed of the moving channel 200.

[0194] In addition, when it is recognized that a plurality of moving boards are not recognized based on a plurality of images 10 acquired through the camera 110 when the robot 100 is mounted on the moving channel 200, the processor 130 may control the suspension 122 of at least one wheel based on a sensing value acquired through a sensor of the robot 100 so that the robot 100 stands upright. In this case, the sensor of the robot 100 may include an inertial measurement unit (IMU) sensor.

[0195] For example, when an object exists in front of the robot 100, the moving plate 220 may not be included in the image obtained through the camera 110. In this case, the processor 130 may not be able to recognize the tilt angle of the moving channel 200. Therefore, when the moving plate 220 of the moving channel 200 is not detected on the plurality of images 10 obtained through the camera 110 when the robot 100 is mounted on the moving channel 200, or the detected moving channel 200 is less than a predetermined number, the processor 130 may recognize the tilt angle of the moving channel 200 through the IMU sensor.

[0196] The IMU sensor is a sensor composed of an accelerometer and a gyroscope, and can sense the posture of the robot and calculate the angle formed by the posture of the robot. In this case, the angle formed by the posture of the robot may correspond to the tilt angle of the moving channel 200. Therefore, the processor 130 may identify an angle for the robot 100 to stand upright (or for the robot 100 to be flush with the floor surface) based on the sensing information (i.e., the angle of the posture of the robot) acquired according to the IMU sensor. The processor 130 may control the robot 100 to stand upright by extending or compressing the suspension 122 connected to at least one wheel based on the identified angle.

[0197] When the robot 100 identifies that there is an object in front of the robot 100 or the distance between the object and the robot 100 is within a predetermined distance based on, for example, a lidar sensor and / or a ToF sensor, the suspension 122 of at least one wheel can be controlled based on the sensing value obtained by the IMU sensor so that the robot 100 stands upright.

[0198] The processor 130 may identify the inclination angle of the moving channel 200 based on the normal vectors of the plurality of moving plates 220 identified through the plurality of images 10, and may use the IMU sensor even when controlling the suspension 122 connected to at least one wheel based on the identified inclination angle of the moving channel 200. Specifically, the processor 130 may control the suspension 122 connected to at least one wheel based on the normal vector of the moving plate located in front of the robot so that the robot 100 stands upright, and then verify whether the robot 100 stands upright correctly based on the sensing value acquired through the IMU sensor. When it is identified that the robot 100 does not stand upright correctly based on the sensing value acquired through the IMU sensor, the processor 130 may identify a compensation value for the suspension 122 using the acquired sensing value. Based on the identified compensation value, the processor 130 may control the suspension 122 connected to at least one wheel again.

[0199] In the following, based on Fig.10 and 11 , a detailed method of aligning the posture of the robot 100 to leave the moving channel 200 will be described.

[0200] When it is identified that the robot 100 is within a predetermined distance from the departure area 4 of the moving channel 200, the processor 130 identifies the moving direction of the moving channel 200 based on the acquired multiple images 10, controls the driver 120 so that the robot 100 faces the departure direction corresponding to the identified moving direction, and controls the driver 120 so that the upright robot 100 moves in the departure direction and leaves the moving channel 200.

[0201] Specifically, the processor 130 may first identify the departure area 4 , and identify the distance between the identified departure area 4 and the robot 100 .

[0202] The exit area 4 refers to an area where a person or robot 100 boarding the moving passage 200 lands after leaving the moving plate 220 of the moving passage 200. For example, the exit area 4 may include a fixed plate 210 installed at the exit of the moving passage 200.

[0203] The exit area 4 may further include a floor surface around the exit of the moving passage 200. For example, the boarding area 3 includes a fixed plate 210 installed at a boarding entrance of the moving passage 200, and may include a floor surface of a predetermined range around the fixed plate 210.

[0204] The processor 130 may identify the departure area 4 based on the map data. Specifically, the departure area 4 of each moving passage 200 may be preset in the map data. Therefore, the processor 130 may identify the position of the departure area 4 based on the map data. The processor 130 may identify the real-time position of the robot 100 on the map data, and identify the distance between the robot 100 and the departure area 4 of the moving passage 200 on which the robot 100 is mounted. When the distance between the position of the robot on the map data and the position of the departure area 4 of the moving passage 200 on which the robot 100 is mounted is within a predetermined distance, the processor 130 may identify that the robot 100 is located within a predetermined distance from the departure area 4 of the moving passage 200.

[0205] In addition, the processor 130 may identify the fixed plate 210 on the image acquired by the camera 110 while the robot 100 is traveling, and identify the distance between the departure area 4 and the robot 100 based on the identified fixed plate 210 .

[0206] For example, on the image acquired by the camera 110 when the robot 100 climbs onto the moving passage 200, only the moving plate 220 of the moving passage 200 located in front of the robot 100 may be detected. However, as the robot 100 gradually approaches the departure area 4 of the moving passage 200, the fixed plate 210 located in the departure area 4 may be detected on the image acquired by the camera 110. Therefore, when the fixed plate 210 is detected on the acquired image, the processor 130 may identify the area including the detected fixed plate 210 as the departure area 4, and identify the distance between the identified departure area 4 and the robot 100.

[0207] Fig.10 is a diagram for describing recognition of a departure area and a distance between the departure area and a robot based on an image acquired by a camera according to one or more embodiments of the present disclosure.

[0208] Specifically, refer to Fig.10 , the fixing plate 210 is not detected in the image 13 acquired at time t6, but the fixing plate 210 is detected in the images 14 and 15 acquired at time t7 and t8. In this case, the processor 130 may identify the distance between the detected fixing plate 210 and the robot 100. The distance between the fixing plate 210 and the robot 100 may be identified based on a depth value of the fixing plate 210 included in an image acquired by a camera 110 (e.g., a depth camera 110 and / or a stereo camera 110) or a sensed value of the distance between the robot 100 and the fixing plate 210 acquired by a sensor. In addition, when the distance between the fixing plate 210 and the robot 100 is within a predetermined distance, the processor 130 may recognize that the robot 100 is located within a predetermined distance from the departure area 4.

[0209] Optionally, as the robot 100 approaches the exit area 4, the processor 130 may recognize that the area of ​​the fixed plate 210 of the moving passage 200 is gradually expanded on the image 10 acquired by the camera 110. Therefore, as the robot 100 approaches the exit area 4 of the moving passage 200, the processor 130 may recognize that the area of ​​the fixed plate 210 is gradually expanded on the image acquired in real time by the camera 110.

[0210] As an example, return to the reference Figure 8 , the area of ​​the fixing plate 210 detected in the image 15 acquired at time t8 (or the ratio of the fixing plate 210) increases more than the area detected in the image 14 acquired at time t7 (or the ratio of the fixing plate 210). In this case, the processor 130 may identify the area whose position and size are changed in the plurality of images 10 acquired by the camera 110 as the departure area 4. Therefore, when the area of ​​the fixing plate 210 is greater than the predetermined range, the processor 130 may recognize that the robot 100 is located within a predetermined distance from the departure area 4.

[0211] Alternatively, the processor 130 may identify the number of the plurality of moving boards 220 identified through the image 10 acquired when the robot 100 boards the moving passage 200, and may identify that the robot 100 is located within a predetermined distance from the departure area 4 based on the number of the identified moving boards 220. Specifically, when the number of the identified moving boards 220 changes or the number of the identified moving boards 220 is less than a predetermined number, the processor 130 may identify that the robot 100 is located within a predetermined distance from the departure area 4.

[0212] Return to reference Figure 8 , the number of moving boards 220 detected in the image 13 acquired at time t6 is 3, the number of moving boards 220 detected in the image 14 acquired at time t7 is 2, and the number of moving boards 220 detected in the image 150 acquired at time t8 is 1. In this case, when the predetermined number is 2, the processor 130 may recognize that the robot 100 is located within the predetermined distance from the departure area 4 at time t8.

[0213] The robot 100 can identify the departure area 4 on the image 10 through a neural network model trained to identify the departure area 4. Specifically, by using a neural network model trained by a plurality of images (i.e., training data) of the landing area 4 of the moving passage 200 including the traveling space where the robot 100 is located, which can be acquired in various landing areas 4, the processor 130 can identify the departure area 4 of the moving passage 200. The neural network model for detecting the departure area 4 included in the image can be implemented as, for example, a convolutional neural network (CNN) model, a fully convolutional neural network (FCN) model, a region having convolutional neural network features (RCNN) model, and / or a YOLO model. The neural network model can be stored in a memory.

[0214] Alternatively, when the identified inclination angle of the moving passage 200 is within a predetermined angle, the processor 130 may recognize that the robot 100 is within a predetermined distance from the departure area 4 of the moving passage 200. In the case of an inclined moving passage 200, the inclination angle may be the smallest in the departure area 4 of the moving passage 200. For example, in the departure area 4 of the moving passage 200, the inclination angle may be 0° so as to be parallel to the moving passage 200 and the floor surface on which the moving passage 200 is installed.

[0215] Therefore, as the robot 100 approaches the departure area 4 of the moving channel 200, the identified inclination angle of the moving channel 200 may also decrease. Therefore, the processor 130 may identify the inclination angle of the moving channel 200 in real time based on the normal vector of the moving plate 220 located in front of the robot 100, and when the identified inclination angle is within a predetermined angle, the processor 130 may recognize that the robot 100 is within a predetermined distance from the departure area 4 of the moving channel 200.

[0216] The processor 130 may recognize that the robot 100 is located in the boarding area 3 by recognizing a column beam installed on the fixing plate 210 of the boarding area 3. For example, when a column beam is recognized on an image acquired by the camera 110 when the robot 100 boards the moving channel 200, the processor 130 may recognize that there is a departure area 4 in front of the robot 100. To this end, the processor 130 may use a neural network model trained to recognize a column beam of the departure area 4.

[0217] The neural network model may be a convolutional neural network (CNN) including a convolutional layer for extracting image feature information and a fully connected layer trained to recognize objects included in the image based on the extracted feature information. The convolutional neural network may be stored in a memory. In addition, the processor 130 may communicate with the column beam of the boarding area 4 through the communication interface of the robot 100 to identify the distance between the robot 100 and the column beam. In this case, when the identified distance is within a predetermined distance, the processor 130 may recognize that the robot 100 is located in the boarding area 3.

[0218] In this manner, the processor 130 may identify the departure area 4 and identify the distance between the identified departure area 4 and the robot 100 through various methods.

[0219] When the robot 100 is identified as being within a predetermined distance from the exit area 4, the processor 130 may identify the moving direction of the moving passage 200 based on the acquired plurality of images 10, and control the driver 120 so that the robot 100 faces the exit direction corresponding to the identified moving direction. The processor 130 may control the driver 120 so that the robot 100, which is maintained in an upright state, moves in the exit direction and leaves the moving passage 200.

[0220] Specifically, the processor 130 may set the ROI 20 including the moving plate 220 of the moving passage 200 on the plurality of images 10 acquired through the camera 110, and identify a position change of the moving plate 220 within the ROI 20. The moving direction of the moving passage 200 in the departure area 4 may be identified based on the identified position change of the moving plate 220.

[0221] In this case, the processor 130 may identify a boundary line between the plurality of moving plates 220 using a straight line detection algorithm, extract a normal vector 40 perpendicular to the identified boundary line, and identify a moving direction of the moving channel 200 in the exit area 4 .

[0222] As a method of recognizing the moving direction of the moving passage 200 in the departure area 4 based on the acquired plurality of images 10 by the processor 130, a method of recognizing the moving direction of the moving passage 200 in the boarding area 3 may be similarly applied. Therefore, a detailed description thereof will be omitted.

[0223] After identifying the moving direction of the moving passage 200, the processor 130 may control the driver 120 so that the robot 100 faces the departure direction corresponding to the identified moving direction. Specifically, the processor 130 may generate control information based on the moving direction of the moving passage 200 in the departure area 4, and rotate the wheels 121 included in the driver 120 based on the generated control information to control the robot 100 so that the robot 100 faces the boarding direction. That is, the processor 130 may control the driver 120 to align the robot 100 in the moving direction of the moving passage 200.

[0224] The processor 130 may control the driver 120 to make the upright robot 100 move in the exit direction and leave the moving passage 200. Specifically, the processor 130 may identify whether the robot 100 is parallel to the floor surface. In this case, as the inclination angle of the moving passage 200 decreases in the exit area 4, the robot 100 within a predetermined distance from the exit area 4 may be parallel to the floor surface.

[0225] The processor 130 may verify the posture of the robot through the IMU sensor. That is, it is possible to identify whether the robot 100 is upright (or parallel to the floor surface) based on the sensing value obtained through the IMU sensor, and when the robot 100 is not upright (or when it is identified that the robot is not parallel to the floor surface), control the suspension 122 connected to at least one wheel based on the sensing value obtained through the IMU sensor. When it is identified that the robot 100 is upright (or when it is identified that the robot 100 is parallel to the floor surface), the processor 130 may control the driver 120 so that the robot 100 moves in the identified exit direction and leaves the moving channel 200.

[0226] In this case, the moving speed of the robot 100 leaving the moving channel 200 may be set based on the moving speed of the moving channel 200. That is, the processor 130 may set the leaving speed to a speed higher than the moving speed of the moving channel 200 by a predetermined ratio. The processor 130 may control the driver 120 to move the robot 100 to the identified leaving direction at the set leaving speed.

[0227] To this end, the processor 130 may identify the moving speed of the mobile channel 200 in the departure area 4. Specifically, the processor 130 sets the ROI 20 including the moving plate 220 of the mobile channel 200 in the plurality of images 10 acquired through the camera 110, and identifies the moving speed of the mobile channel 200 based on the position change of the moving plate 220 in the ROI 20 and the frame rate of the plurality of images 10.

[0228] In this case, the processor 130 may use a straight line detection algorithm to identify the boundary line between the plurality of moving plates 220, and track the identified boundary line to identify the position change of the boundary line. As a method for the processor 130 to identify the moving speed of the moving passage 200 in the departure area 4 based on the acquired plurality of images 10, a method for identifying the moving speed of the moving passage 200 in the boarding area 3 may be similarly applied. Therefore, a detailed description thereof will be omitted.

[0229] The processor 130 may control the suspension 122 connected to at least one of the plurality of wheels 121 so that the robot 100 tilts at an angle corresponding to the moving speed of the moving passage 200 in the departure area 4, and control the driver 120 so that the tilted robot 100 moves in the departure direction and leaves the moving passage 200. Specifically, the processor 130 may stretch or compress the suspension 122 connected to at least one wheel by considering the friction force in the departure area 4 so that the robot 100 tilts in the direction opposite to the departure direction. For example, the processor 130 may stretch the suspensions (i.e., the first suspension 122-1 and the second suspension 122-2) connected to the wheels 121 (i.e., the first wheel 121-1 and the second wheel 121-2) located on the departure direction side of the robot 100 to control the robot 100 so that the robot 100 tilts in the direction opposite to the departure direction.

[0230] In this regard, a method of identifying the tilt angle of the robot 100 based on the moving speed of the moving channel 200 in the boarding area 3 and a method of controlling the suspension 122 connected to at least one wheel to control the tilt of the robot 100 may be similarly applied, and therefore, a detailed description thereof will be omitted.

[0231] The processor 130 may recognize the posture of the robot 100 as an upright posture or a tilted posture according to the driving mode of the robot 100 in the departure area 4. For example, when the robot 100 stops in the departure area 4 after leaving the moving passage 200, the suspension 122 connected to at least one wheel may be controlled so that the posture of the robot is tilted at an angle corresponding to the moving speed of the moving passage 200. However, when the robot 100 moves in the departure area 4 after leaving the moving passage 200, the suspension 122 connected to at least one wheel may be controlled so that the robot 100 stands upright.

[0232] When it is recognized that an object exists in the exit area 4 , the processor 130 may control the driver 120 so that the robot 100 moves in a direction opposite to the exit direction.

[0233] Specifically, the processor 130 may identify whether the object 410 exists in the departure area 4 based on the image acquired through the camera 110. That is, when the departure area 4 and the object 410 in the departure area 4 are detected in the acquired image, the processor 130 may identify that the object 410 exists in the departure area 4. Alternatively, the processor 130 may identify whether the object 410 exists in the departure area 4 located in front of the robot 100 through a sensor (e.g., a lidar sensor and / or a ToF sensor).

[0234] When the robot 100 recognizes that the object 410 exists in the departure area 4, the processor 130 may recognize the distance between the object 410 existing in the departure area 4 and the robot 100. When it is recognized that the recognized distance is less than a predetermined distance, the processor 130 may control the plurality of wheels 121 of the driver 120 so that the robot 100 moves in a direction opposite to the recognized departure direction. Specifically, the processor 130 may control the plurality of wheels 1210 to rotate in a direction opposite to the moving direction of the moving channel 200.

[0235] In this case, the processor 130 may control the driver 120 so that the robot 100 moves in the direction opposite to the departure direction at the same speed as the moving speed of the moving passage 200 in the identified departure area 4. Therefore, the robot 100 may maintain its position on the moving passage 200. This may produce the same effect as a person walking in place in the moving passage 200. When it is recognized that the robot 100 does not exist in the departure area 4 (for example, when it is recognized that an object existing in the departure area 4 disappears), the processor 130 may control the driver 120 so that the robot 100 moves in the departure direction and leaves the moving passage 200.

[0236] Fig.11 1 is a diagram for explaining that, when an object exists in a departure area, the driver 120 is controlled so that the robot 100 moves in a direction opposite to a departure direction according to one or more embodiments of the present disclosure.

[0237] Reference Fig.11 , the processor 130 may detect that a person 410 exists in the departure area 4. In this case, the processor 130 may control the driver 120 so that the robot 100 moves in a direction opposite to the departure direction at the same speed v as the moving speed v of the moving passage 200 in the departure area 4. In this way, the robot 100 may maintain its position on the moving passage 200 until the person 410 existing in the departure area 4 disappears. Therefore, it is possible to avoid a collision with the person 410 existing in the departure area 4 while staying on the moving passage 200.

[0238] When the processor 130 controls the driver 120 to move the robot 100 in a direction opposite to the departure direction by recognizing that the object exists in the departure area 4 , the processor 130 may recognize whether another object 420 exists behind the robot 100 .

[0239] Specifically, return to reference Fig.11 , the processor 130 may rotate an image acquired by the camera 110 disposed at the rear of the robot 100 or a sensor disposed at the rear or a sensor disposed at the robot 100 to recognize another object 420 located behind the robot 100. In this case, when the processor 130 controls the driver 120 so that the robot 100 moves in a direction opposite to the departure direction, the processor 130 may recognize a distance between the another object 420 located behind the robot 100 and the robot 100.

[0240] When the distance between another object 420 located behind the robot 100 and the robot 100 is within a predetermined distance, the processor 130 may send a signal requesting to stop the moving channel 200 to the server controlling the moving channel 200 through the communication interface of the robot 100. Alternatively, the processor 130 may directly send a control signal to stop the operation of the moving channel 200 to the moving channel 200 through the communication interface.

[0241] Fig.12 is a detailed configuration diagram of a robot according to one or more embodiments of the present disclosure.

[0242] Reference Fig.12 The robot 100 includes a camera 110, a driver 120, a display 140, a memory 150, a sensor 160, a speaker 170, a user interface 180, and a communication interface 190. Fig.12 The components shown are Figure 2 Detailed description of components where components are shown to overlap.

[0243] The display 140 may display various types of visual information. For example, the display 140 may display information about map data, a travel path of the robot 100 , and / or a moving direction and moving speed of the escalator 200 .

[0244] To this end, the display 140 may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, the display 140 may be implemented as various types of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a light emitting diode (LED), a micro LED, a mini LED, a plasma display panel (PD), a quantum dot (QD) display, and a quantum dot light emitting diode (QLED).

[0245] The display 140 may include a driving circuit and / or a backlight unit implemented in a form such as a-si TFT, low temperature polysilicon (LTPS) TFT, and / or an organic TFT (OTFT). The display 140 may be implemented as, for example, a touch screen coupled to a touch sensor, a flexible display, a rollable display, a 3D display, and / or a display to which a plurality of display modules are physically connected.

[0246] In addition, the display 140 may be implemented as a touch screen together with the touch panel.

[0247] Therefore, the display unit 140 may be used as an output unit for outputting information between the robot 100 and the user, and simultaneously as an input unit for providing an input interface between the robot 100 and the user.

[0248] The memory 150 may store data. For example, map data regarding a travel space in which the robot 100 is located may be stored in the memory 150 .

[0249] The memory 150 may be implemented in the form of a memory embedded in the robot 100 or in the form of a memory detachable from the robot 100 according to the purpose of data storage. For example, data for driving the robot 100 may be stored in the memory embedded in the robot 100, and data for extended functions of the robot 100 may be stored in a memory attachable to and detachable from the robot 100.

[0250] The memory embedded in the robot 100 may include, for example, at least one of a volatile memory (e.g., dynamic random access memory (DRAM), static RAM (SRAM), and / or synchronous dynamic RAM (SDRAM)), a non-volatile memory (e.g., one-time programmable read-only memory (OPTROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM, and / or a flash ROM, a flash memory (e.g., a NAND flash memory and / or a NOR flash memory), a hard disk drive, and a solid-state drive (SSD)).

[0251] In addition, the memory 300 that is removable from the robot 100 may be implemented in the form of a memory card (e.g., Compact Flash (CF), Secure Digital (SD), Micro Secure Digital (Micro-SD), Mini Secure Digital (Mini-SD), Extreme Digital (xD), MultiMedia Card (MMC)), and / or an external memory that can be connected to a USB port (e.g., a USB memory)).

[0252] According to an example, the memory 150 may store information about a plurality of neural network (or artificial intelligence) models. For example, the memory 150 may store a neural network model trained to recognize a landing area in an image acquired by the camera 110, and a neural network trained to recognize a departure area 4 in an image acquired by the camera 110.

[0253] Storing information about the neural network model may mean storing various types of information related to the operation of the neural network model, for example, information about at least one layer included in the neural network model, information about parameters and / or bias used in each of the at least one layer. However, it goes without saying that the information about the neural network model may be stored in the internal memory of the processor 130 according to the implementation form of the processor 130. For example, when the processor 130 is implemented as dedicated hardware, the information about the neural network model may be stored in the internal memory of the processor 130.

[0254] The robot 100 includes at least one sensor 160. In addition to the above-mentioned sensors for detecting objects around the robot 100 (e.g., a lidar sensor and / or a ToF sensor) and sensors for detecting the posture of the robot (e.g., an IMU sensor), the at least one sensor 160 may further include at least one of a gesture sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor (e.g., a red, green, blue (RGB) sensor), a biosensor, a temperature / humidity sensor, an illumination sensor, and an ultraviolet (UV) sensor.

[0255] The speaker 170 may output a sound signal to the outside of the robot 100. The speaker 170 may output, for example, multimedia reproduction, recording reproduction, various notification sounds, and / or voice messages. The robot 100 may include an audio output device such as the speaker 170, or may include an output device such as an audio output terminal. Specifically, the speaker 170 may provide, for example, acquired information, information processed / generated based on the acquired information, a response result to a user's voice, and / or an operation result in the form of voice. For example, when it is recognized that the robot 100 is located within the departure area 4 and a predetermined distance of the moving channel 200 and there is an object in the departure area 4, the processor 130 may output a warning sound through the speaker 170.

[0256] The user interface 180 is a component for the robot 100 to interact with the user, and the processor 130 may receive various types of information through the user interface 180, such as control information of the robot 100, travel space information where the robot 100 is located (e.g., map data of the travel space), and information about the moving passage 200. The user interface 180 may include at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, or a microphone, but is not limited thereto.

[0257] The communication interface 190 can send and receive various types of content. For example, the communication interface 190 can receive or send signals from an external device (e.g., a user terminal), an external storage medium (e.g., a USB memory), and / or an external server (e.g., a web hard disk) in a streaming or downloading method through communication methods such as AP-based Wi-Fi (wireless LAN network), Bluetooth, Zigbee, wired / wireless local area network (LAN), wide area network (WAN), Ethernet, IEEE 139, High Definition Multimedia Interface (HDMI), Universal Serial Bus (USB), Mobile High Definition Link (MHL), Audio Engineering Society / European Broadcasting Union (AES / EBU), optical and coaxial, etc.

[0258] The processor 130 may send a signal requesting to stop driving the mobile channel 200 to the control server 500 of the mobile channel 200 through the communication interface 190. Alternatively, the processor 130 may send a control signal for stopping driving the mobile channel 200 to the mobile channel 200 through the communication interface 190.

[0259] In addition, the processor 130 may communicate with the user terminal through the communication interface 190. In this case, the processor 130 may receive location information of the user terminal communicated through the communication interface 190, and control the driver 120 so that the robot 100 tracks the user terminal.

[0260] Fig.13 is a flow chart schematically illustrating a method for controlling a robot to board a moving passage according to one or more embodiments of the present disclosure.

[0261] In the method for controlling the robot 100, the processor 130 identifies whether the robot 100 is located in the landing area 3 of the moving passage 200 (S1310).

[0262] The processor 130 identifies the moving direction and moving speed of the moving channel 200 based on the plurality of images 10 of the moving channel 200 acquired through the camera 110 (1320). In this case, the processor 130 may determine the boundary line between the plurality of plates of the moving channel 200 based on the plurality of images 10 acquired through the camera 110 when the robot 100 is located in the boarding area 3, and identify the moving direction of the moving channel 200 based on the normal vector of the identified boundary line. The processor 130 may identify the moving speed of the moving channel 200 based on the moving speed of the identified boundary line.

[0263] The processor 130 identifies the moving direction and the moving speed of the moving channel 200, and then controls the driver 120 of the robot 100 so that the robot 100 faces the boarding direction corresponding to the identified moving direction (S1330).

[0264] The processor 130 controls the suspension 122 connected to at least one of the multiple wheels 121 included in the driver 120 so that the robot 100 tilts at an angle corresponding to the identified moving speed, and then controls the driver 120 so that the tilted robot 100 moves in a boarding direction and boards the moving channel 200 (S1350).

[0265] In this case, each of the plurality of wheels 121 is connected to the main body of the robot 100 through a suspension 122 .

[0266] Fig.14 1 is a flowchart schematically illustrating a method of controlling the robot when the robot 100 climbs onto the moving channel 200 according to one or more embodiments of the present disclosure.

[0267] Fig.14 Steps S1410, S1420, S1430, S1440 and S1450 shown in Fig.13 1310, S1320, S1330, S1340, and S1350 described in. Therefore, a detailed description thereof will be omitted.

[0268] Reference Fig.14 , when it is recognized that the robot 100 has boarded the moving passage 200, the processor 130 recognizes a plurality of boards of the moving passage 200 based on a plurality of images 10 of the moving passage 200 acquired through the camera 110 (S1460).

[0269] The processor 130 identifies the inclination angle of the moving channel 200 based on the normal vector of at least one of the identified multiple panels (S1470). In this case, the processor 130 may identify at least one of the multiple panels based on the moving speed of the moving channel 200 and the response speed of the suspension 122. That is, the processor 130 may select a target panel for extracting a normal vector for identifying the inclination angle of the moving channel 200 from among the multiple panels based on the moving speed of the moving channel 200 and the response speed of the suspension 122.

[0270] After identifying the inclination angle of the moving channel 200, the processor 130 controls the suspension 122 of at least one wheel based on the identified inclination angle so that the robot 100 stands upright (S1480). For example, the state in which the robot 100 stands upright may be a state in which the robot 100 is parallel to the floor surface on which the moving channel 200 is installed.

[0271] When it is recognized that a plurality of boards are not recognized based on the plurality of images 10 acquired through the camera 110 when the robot 100 is mounted on the moving channel 200, the processor 130 may control the suspension 122 of at least one wheel based on the sensing value acquired through the sensor of the robot 100 so that the robot 100 stands upright. In this case, the sensor of the robot 100 may include an inertial measurement unit (IMU) sensor. That is, the processor 130 may recognize the posture of the robot based on the sensing value acquired through the IMU sensor, and control the suspension 122 of at least one wheel so that the recognized posture of the robot stands upright.

[0272] Fig.15 is a flowchart schematically illustrating a method of controlling the robot 100 to leave the moving channel 200 according to one or more embodiments of the present disclosure.

[0273] Fig.15 Steps S1505, S1510, S1515, S1520, S1525, S1530, S1535, and S1540 shown in Fig.14 1410, S1420, S1430, S1440, S1450, S1460, S1470, and S1480 described in. Therefore, a detailed description thereof will be omitted.

[0274] When it is recognized that the robot 100 is located within a predetermined distance from the departure area 4 of the moving passage 200, the processor 130 recognizes the moving direction of the moving passage 200 based on the acquired plurality of images 10 (S1545). In this case, when the tilt angle recognized based on the normal vector of the board is within a predetermined angle, the robot 100 is within a predetermined distance from the departure area 4 of the moving passage 200.

[0275] The processor 130 controls the driver 120 so that the robot 100 faces the departure direction corresponding to the recognized moving direction (S1550).

[0276] After controlling the driver 120 so that the robot 100 faces the identified moving direction, the processor 130 controls the driver 120 so that the upright robot 100 moves in the exit direction and leaves the moving passage 200 (S1555). That is, the processor 130 may control the driver 120 so that the robot 100 parallel to the floor surface on which the moving passage 200 is installed moves in the exit direction and leaves the moving passage 200.

[0277] In this case, when it is recognized that an object exists in the exit area 4, the processor 130 may control the driver 120 so that the robot 100 moves on the moving channel 200 in a direction opposite to the exit direction.

[0278] In the above description, steps S1310 to S1350, S1410 to S1480, and S1505 to S1555 may be further divided into additional steps or combined into fewer steps. In addition, if necessary, some steps may be omitted, and the order between operations may be changed.

[0279] The above method can be implemented in the form of an application that can be installed in an existing robot. Optionally, a learning neural network (or deep learning neural network) based on deep learning (i.e., a learning network model) can be used to perform the above method. In addition, the above method can be implemented only by software upgrades or hardware upgrades of existing robots. In addition, the above various embodiments can be performed by an embedded server set in the robot or a server outside the robot.

[0280] The various embodiments described above may be implemented by software including instructions stored in a machine-readable storage medium (e.g., a computer-readable storage medium). The machine may be a device that calls stored instructions from a storage medium and can operate according to the called instructions, and may include a display device (e.g., display device A) according to the disclosed embodiment. In the case where a processor executes a command, the processor may directly execute a function corresponding to the command, or other components may execute functions corresponding to the command under the control of the processor. The command may include code created or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" means that the storage medium is tangible and does not include a signal, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

[0281] In addition, the above methods according to various embodiments may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be in a form that can be read by a machine or through an application store (e.g., PlayStore). TM ) in the form of a storage medium that can be read online (e.g., a compact disk read-only memory (CD-ROM)). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored in a storage medium such as a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

[0282] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may include a single entity or multiple entities, and some of the corresponding subcomponents described above may be omitted, or other subcomponents may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each corresponding component before integration. The operations performed by the modules, programs, or other components according to the various embodiments may be performed in a sequential manner, in a parallel manner, in an iterative manner, or in a heuristic manner, and at least some operations may be performed or omitted in a different order, or other operations may be added.

[0283] Although exemplary embodiments have been shown and described above, the embodiments are not limited to the above-mentioned specific exemplary embodiments, but various modifications can be made by those skilled in the art without departing from the gist of the present disclosure disclosed in the appended claims. These modifications should also be understood to fall within the scope and spirit of the present disclosure.

Claims

1. A robot comprising: camera; a drive, comprising: a plurality of wheels and a suspension for connecting each of the plurality of wheels to a body of the robot; The processor is configured as: Based on the robot being identified as being located in an entrance area of ​​the moving channel, identifying a moving direction of the moving channel and a moving speed of the moving channel based on a plurality of images of the moving channel received from the camera, controlling the actuator so that the robot faces an entry direction corresponding to the identified movement direction; controlling a suspension connected to at least one of the plurality of wheels so that the robot tilts at an angle corresponding to the identified movement speed; and The driver is controlled so that the tilted robot moves according to the entry direction and enters the movement channel.

2. The robot according to claim 1, wherein: The processor is further configured to: recognizing boundary lines between a plurality of plates of the moving passage based on the plurality of images received from the camera when the robot is located in an entrance area; identifying a moving direction of the moving channel based on a normal vector of the identified boundary line; and A moving speed of the moving channel is identified based on the moving speed of the identified boundary line.

3. The robot according to claim 1, wherein: The processor is configured to: based on the robot being identified as entering the moving passage, identifying a plurality of plates of the moving passage based on the plurality of images of the moving passage received from the camera; identifying an inclination angle of the moving channel based on a normal vector of at least one of the identified plurality of plates; and The suspension connected to the at least one wheel among the plurality of wheels is controlled based on the identified tilt angle so that the robot stands upright.

4. The robot according to claim 3, wherein: The processor is configured to identify the at least one panel of the plurality of panels based on a moving speed of a moving channel and a response speed of the suspension.

5. The robot according to claim 3, in, The robot further comprises: a sensor; Wherein, the processor is further configured to: based on the fact that the multiple boards are not identified based on the multiple images received from the camera when the robot enters the moving channel, control the suspension connected to at least one of the multiple wheels based on the sensing value received from the sensor, so that the robot stands upright.

6. The robot according to claim 3, in, The processor is configured to: Based on the robot being identified as being located within a predetermined distance from an exit area of ​​the moving passage, re-identifying a moving direction of the moving passage based on the plurality of received images; controlling the driver so that the robot faces an exit direction corresponding to the re-identified moving direction; and The drive is controlled so that the upright robot moves in the exit direction and leaves the moving channel.

7. The robot according to claim 6, wherein: Based on the object being identified as being located in the exit area, the processor controls the driver to move the robot in a direction opposite to the exit direction.

8. The robot according to claim 6, wherein: The processor is configured to: Based on the identified tilt angle being identified as being within a predetermined angle range, it is identified that the robot is within the predetermined distance from the exit area of ​​the moving passage.

9. The robot according to claim 1, wherein: The processor is further configured to: identifying a width of the movement passage based on the plurality of images received from the camera when the robot is located in the entry area; determining an entrance position of the moving passage of the robot based on the identified width of the moving passage; and The drive is controlled so that the robot faces the entry direction corresponding to the identified movement direction at the determined entry position.

10. A method for controlling a robot, comprising: Based on the robot being identified as being located in an entrance area of ​​the moving passage, identifying a moving direction of the moving passage and a moving speed of the moving passage based on a plurality of images of the moving passage received from a camera; controlling a driver of the robot so that the robot faces an entry direction corresponding to the identified movement direction; controlling a suspension of the actuator so that the robot tilts at an angle corresponding to the identified moving speed, wherein the suspension is connected to at least one wheel of a plurality of wheels included in the actuator; and Controlling the driver so that the tilted robot moves according to the entry direction and enters the moving channel; Wherein, each of the plurality of wheels is connected to the body of the robot via the suspension.

11. The method of claim 10, wherein: Identifying a moving direction of the moving channel and a moving speed of the moving channel includes: identifying, when the robot is located in the entrance area, boundary lines between a plurality of plates of the moving passage based on the plurality of images received from the camera; identifying a moving direction of the moving channel based on a normal vector of the identified boundary line; and A moving speed of the moving channel is identified based on the moving speed of the identified boundary line.

12. The method of claim 10, further comprising: based on the robot being identified as entering the moving passage, identifying a plurality of plates of the moving passage based on the plurality of images of the moving passage received from the camera; identifying an inclination angle of the moving channel based on a normal vector of at least one of the identified plurality of plates; and The suspension connected to the at least one wheel among the plurality of wheels is controlled based on the identified tilt angle so that the robot stands upright.

13. The method of claim 12, further comprising: The at least one panel of the plurality of panels is identified based on a moving speed of the moving channel and a response speed of the suspension.

14. The method of claim 12, further comprising: Based on the fact that the multiple boards are not recognized based on the multiple images received from the camera when the robot enters the moving channel, the suspension connected to at least one of the multiple wheels is controlled based on a sensing value obtained by a sensor of the robot, so that the robot stands upright.

15. The method of claim 12, further comprising: re-identifying a moving direction of the moving passage based on the plurality of received images based on the robot being identified as being located within a predetermined distance from an exit area of ​​the moving passage; controlling the driver so that the robot faces an exit direction corresponding to the re-identified moving direction; The drive is controlled so that the upright robot moves in the exit direction and leaves the moving channel.

Images