Stair-traversing robot and control method therefor

Abstract

A stair-traversing robot control method is disclosed. According to one embodiment of the present disclosure, the stair-traversing robot control method is provided, the control method comprising the steps of: generating a first front view image by capturing the front view of the stair-traversing robot; identifying the shape of a stair in the first front view image; moving the stair-traversing robot forward toward the stair on the basis of the shape of the stair such that the stair-traversing robot faces the stair; generating a second front view image by capturing the front view of the stair-traversing robot; generating a third front view image by removing noise from the second front view image; identifying the edge of the stair in the third front view image; calculating the angle between the edge of the stair and the moving direction of the stair-traversing robot; aligning the stair-traversing robot on the basis of the angle such that the edge of the stair is perpendicular to the moving direction of the stair-traversing robot; and controlling a driving unit of the stair-traversing robot such that the stair-traversing robot traverses the stair.

Description

Stair-climbing robot and its control method

[0001] The present disclosure relates to a stair-climbing robot and a control method thereof.

[0002] The material described in this section merely provides background information for the present disclosure and does not constitute prior art.

[0003] In the event of a disaster, such as a building collapse, in a specific structure, robots can be deployed in place of humans to investigate the interior. These robots then roam the disaster site, gathering information about the disaster. When deployed inside a building, robots require stable stair-climbing capabilities. If a robot cannot navigate stairs, it can only gather limited information, significantly reducing its practicality.

[0004] Meanwhile, a human operator can use the robot's onboard camera to gain a view of the disaster site and control the robot. However, the camera's field of view can be distorted by variables such as the camera's angle of view and ambient lighting. Controlling the robot using this distorted view requires a significant level of skill on the part of the human operator.

[0005] Especially when a robot navigates stairs, the robot and the stairs must maintain a certain angle. For example, the robot's direction of travel and the edge of the stairs must be perpendicular to each other. Since it's difficult for a human operator, who uses a camera to secure a field of view, to precisely control the robot, a method to supplement the human operator's control during stair navigation is required.

[0006] Accordingly, the present disclosure is intended to solve these problems, and its main purpose is to provide a control method that enables stable stair climbing of a robot.

[0007] In addition, the main purpose of the present disclosure is to provide a control method that can be applied to various types of robots with different specifications.

[0008] In addition, the present disclosure has a main purpose of providing a robot capable of climbing stairs and a device for controlling the staircase climbing robot.

[0009] According to one embodiment of the present disclosure for achieving the above object, a control method for a stair-climbing robot is provided, comprising: a step of photographing the front of the stair-climbing robot to generate a first front image; a step of identifying a shape of stairs in the first front image; a step of moving the stair-climbing robot toward the stairs based on the shape of the stairs identified in the first front image; a step of photographing the front of the stair-climbing robot to generate a second front image; a step of removing noise from the second front image to generate a third front image; a step of identifying an edge of the stairs in the third front image; a step of calculating an angle between the edge of the stairs and a moving direction of the stair-climbing robot; a step of aligning the stair-climbing robot so that the edge of the stairs and the moving direction of the stair-climbing robot are perpendicular to each other based on the angle; and a step of controlling a driving unit of the stair-climbing robot to cause the stair-climbing robot to navigate the stairs.

[0010] According to one embodiment of the present disclosure for achieving the above object, a stair-climbing robot is provided, including: a camera for photographing the front of the stair-climbing robot; a driving unit for moving the stair-climbing robot; an inertial measurement device for detecting a yaw angle and a pitch angle of the stair-climbing robot; and a control unit, wherein the control unit photographs the front of the stair-climbing robot to generate a first front image, identifies the shape of stairs in the first front image, and moves the stair-climbing robot forward of the stairs so that the stair-climbing robot faces the stairs based on the shape of the stairs; photographs the front of the stair-climbing robot to generate a second front image, removes noise from the second front image to generate a third front image, identifies an edge of the stairs in the third front image, calculates an angle between the edge of the stairs and a moving direction of the stair-climbing robot, and based on the angle, aligns the stair-climbing robot so that the edge of the stairs and the moving direction of the stair-climbing robot are perpendicular to each other, and controls the driving unit so that the stair-climbing robot navigates the stairs.

[0011] According to one embodiment of the present disclosure for achieving the above object, there is provided a device mounted on a robot including a driving unit, comprising: a memory; and a control unit, wherein the control unit is configured to photograph the front of the robot to generate a first front image, identify the shape of stairs in the first front image, and move the robot forward toward the stairs so that the robot faces the stairs based on the shape of the stairs, photograph the front of the robot to generate a second front image, remove noise from the second front image to generate a third front image, identify an edge of the stairs in the third front image, calculate an angle between the edge of the stairs and a moving direction of the robot, and align the robot so that the edge of the stairs and the moving direction of the robot are perpendicular to each other based on the angle, and control the driving unit to perform a process of the robot traveling along the stairs.

[0012] As described above, the present embodiment enables a robot to navigate stairs stably. Specifically, unlike conventional technologies, the present invention utilizes images from cameras, which are standard on most robots, to recognize stairs and extract the edges of the stairs, without requiring various types of sensors. Furthermore, the present invention aligns the stair-climbing robot with respect to the edges of the stairs, enabling stable staircase navigation.

[0013] In addition, the control method provided in the present disclosure can be applied to various types of robots with different specifications.

[0014] FIG. 1 is a functional block diagram of a stair-climbing robot according to one embodiment of the present disclosure.

[0015] FIG. 2 is a drawing illustrating a first front image according to one embodiment of the present disclosure.

[0016] FIG. 3 is a diagram illustrating a second front image and a third front image according to one embodiment of the present disclosure.

[0017] FIGS. 4 to 7 are flowcharts illustrating a control method of a stair-climbing robot according to one embodiment of the present disclosure.

[0018] Hereinafter, some embodiments of the present disclosure will be described in detail using exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, when describing the present disclosure, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present disclosure.

[0019] In describing the components of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by the terms.

[0020] When a component is described as being 'connected', 'coupled' or 'connected' to another component, it should be understood that the component may be directly connected or connected to that other component, but that another component may also be 'connected', 'coupled' or 'connected' between the components.

[0021] Throughout the specification, when a part is said to 'include' or 'have' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0022] The terms ‘unit’, ‘module’, etc., used in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0023] Unless stated otherwise, the description of any one embodiment is intended to be applicable to other embodiments.

[0024] The following description of the invention, together with the accompanying drawings, is intended to explain exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

[0025] FIG. 1 is a functional block diagram of a stair-climbing robot according to one embodiment of the present disclosure.

[0026] Referring to FIG. 1, the stair climbing robot (1) includes all or part of a sensor unit (100), a control unit (130), a driving unit (150), and a memory (170).

[0027] The sensor unit (100) may include a camera (101). The camera (101) is configured to capture the front of the stair-climbing robot (1) and generate a plurality of front images (180). According to one embodiment, the camera (101) may be an optical camera. According to one embodiment, the camera (101) may be an RGB camera. According to one embodiment, the camera (101) may be an RGBD camera. According to one embodiment, the camera (101) may be a stereo camera. The camera (101) is not limited to a specific type of camera. For example, the camera (101) may be a thermal imaging camera.

[0028] According to one embodiment, the camera (101) can capture the front in real time and generate a front image (180) in real time.

[0029] In order to explain the front image (180) by distinguishing it according to the point in time at which the camera (101) generates the front image (180), a first front image (180A) and a second front image (180B) are defined.

[0030] According to one embodiment, the camera (101) captures a forward image to generate a first forward image (180A). Among the forward images (180), the forward image (180) until the step S420 described below is performed is referred to as the first forward image (180A). In other words, the first forward image (180A) refers to the forward images generated up to the point where the stair-climbing robot (1) reaches the front of the stairs.

[0031] According to one embodiment, the camera (101) captures a forward image to generate a second forward image (180B). Among the forward images (180), the forward image (180) captured by the camera (101) after performing the S420 process described below is referred to as a second forward image (180B). In other words, the second forward image (180B) refers to forward images captured when the stair-climbing robot (1) is facing the stairs.

[0032] According to one embodiment, the front image (180) captured by the camera (101) can be stored in the memory (170).

[0033] The sensor unit (100) may include an inertial measurement device (102). The inertial measurement device (102) includes all or part of an accelerometer, a gyroscope, and a magnetometer. The motion state and posture of the stair-climbing robot (1) can be tracked and monitored using the inertial measurement device (102).

[0034] According to one embodiment, the inertial measurement device (102) can detect the yaw angle, roll angle, and pitch angle of the stair climbing robot (1).

[0035] The driving unit (150) can move the stair-climbing robot (1) according to the control signal of the control unit (130) and control the posture of the stair-climbing robot (1).

[0036] According to one embodiment, the driving unit (150) may include a continuous track for moving the stair-climbing robot (1). The stair-climbing robot (1) according to the present disclosure is not limited to a robot including a continuous track. For example, it may be a robot capable of bipedal or quadrupedal walking. For example, it may be a two-wheel drive robot or a four-wheel drive robot including wheels.

[0037] The memory (170) may store a front image (180) generated by the camera (101). According to one embodiment, the memory (170) may store a first front image (180A) to a third front image (180C) to be described later. The third front image (180C) refers to an image generated by extracting only an image of stairs from a portion of the second front image (180B). The control unit (130) may generate the third front image (180C) by extracting only an image of stairs from a portion of the second front image (180B) in order to reduce noise or environmental influences.

[0038] A look-up table may be stored in the memory (170). In the look-up table, the range of yaw angles and pitch angles required for the stair-climbing robot (1) when climbing stairs are indexed according to the height of the stairs, the slope of the stairs, the width of the stairs, and the vertical width of the stairs. Here, the horizontal width of the stairs is the width in the direction perpendicular to the vertical width of the stairs. The horizontal width of the stairs is the width in the direction parallel to the edge of the stairs.

[0039] A stair-climbing robot (1) can perform posture control of the stair-climbing robot (1) based on a lookup table when climbing stairs. By performing posture control, the direction of movement of the stair-climbing robot (1) can be perpendicular to the edge of the stairs, and the stair-climbing robot (1) can stably climb without deviating from the stairs.

[0040] The control unit (130) generates a control signal for movement and stair climbing of the stair climbing robot (1). Based on the control signal of the control unit (130), the driving unit (150) can move the stair climbing robot (1). The control unit (130) can control the posture of the stair climbing robot (1) based on a yaw angle, a roll angle, and a pitch angle. The control unit (130) can cause the stair climbing robot (1) to approach the stairs. The control unit (130) can align the stair climbing robot (1) with respect to the stairs. The control unit (130) can control the posture of the stair climbing robot (1) so that it can stably climb the stairs.

[0041] FIG. 2 is a drawing illustrating a first front image according to one embodiment of the present disclosure.

[0042] FIG. 3 is a diagram illustrating a second front image and a third front image according to one embodiment of the present disclosure.

[0043] FIGS. 4 to 7 are flowcharts illustrating a control method of a stair-climbing robot according to one embodiment of the present disclosure.

[0044] Referring to FIGS. 2 to 7, a stair-climbing robot (1) according to one embodiment of the present disclosure can stably navigate stairs. The stair-climbing robot (1) can sequentially perform the following steps: recognizing a staircase as a navigation target, approaching the staircase, aligning the robot so that its direction of travel is perpendicular to the staircase after approaching the staircase, and then navigating the staircase after alignment. This will be described in detail below.

[0045] The stair-climbing robot (1) can capture a forward image to generate a first forward image (180A) (S400). Specifically, the camera (101) can capture a forward image in real time to generate multiple first forward images (180A). The generated first forward images (180A) can be stored in the memory (170).

[0046] A stair-climbing robot (1) can identify the shape of stairs in a first forward image (180A) (S410). According to one embodiment, the control unit (130) can identify the shape of stairs in the first forward image (180A) using an object recognition algorithm. The control unit (130) can identify the shape of stairs in the first forward image using a first AI engine. The first AI engine refers to an AI engine trained to identify the shape of stairs in an image. The first AI engine learns stairs using photos of multiple stairs, thereby having the effect of reliably recognizing stairs regardless of the shape, color, size, etc. of the stairs.

[0047] The control unit (130) can perform the S420 process described below based on the shape of the stairs (A0) identified in the first front image.

[0048] The stair-driving robot (1) can move forward toward the stairs (S420). Specifically, the control unit (130) can control the stair-driving robot (1) to move toward the stairs based on the shape of the stairs (A0) identified in the first forward image. In this case, the stair-driving robot (1) moves forward toward the stairs under the control of the control unit (130). In order to determine whether the stair-driving robot (1) is moving toward the stairs, the control unit (130) determines whether the center (C1) of the first area is located within the allowable area (ta). This will be described later.

[0049] The S420 process is explained in more detail.

[0050] The control unit (130) can detect the exact center (C0) of the first front image (S500). The control unit (130) can display the shape (A0) of the stairs identified in the first front image as a first area (A1) on the first front image (180A) (S510). The control unit (130) can detect the exact center (C1) of the first area (S520).

[0051] The control unit (130) can calculate the distance between the exact center (C0) of the first forward image and the exact center (C1) of the first region (S530). The control unit (130) can determine whether the distance between the exact center (C0) of the first forward image and the exact center (C1) of the first region exceeds a preset value (S540). According to one embodiment, the preset value is a value for positioning the exact center (C1) of the first region within the allowable area (ta), which may vary depending on the size or shape of the allowable area (ta). The allowable area (ta) refers to a specific area formed to include the exact center (C0) of the first forward image (Fig. 2 (b)). The size, shape, etc. of the allowable area (ta) may vary depending on the type of robot, the specifications of the robot, the type of site, etc. For example, the allowable area (ta) may have a circular shape, unlike what is disclosed in the drawing.

[0052] The control unit (130) can determine whether the distance between the exact center (C0) of the first front image and the exact center (C1) of the first region is greater than or less than a preset value. Meanwhile, the process S420 aims to place the stair-driving robot (1) in front of the stairs so that the stair-driving robot (1) faces the stairs. If the exact center (C1) of the first region is located within the allowable area (ta), the control unit (130) can determine that the stair-driving robot (1) is facing the stairs. That is, the control unit (130) can compare the size of the distance with a preset value to determine whether the exact center (C1) of the first region is located within the allowable area (ta). If the control unit (130) determines that the stair-driving robot (1) is facing the stairs, the control unit (130) can perform the process S430 described below.

[0053] In the case where the allowable area (ta) is a square shape as shown in FIG. 2, the control unit (130) can also calculate the separation distances separately for the horizontal and vertical directions. For example, the separation distance between the center (C0) of the first front image and the center (C1) of the first region can be calculated by dividing it into the horizontal separation distance and the vertical separation distance. For example, as shown in FIG. 2, when the horizontal separation distance (d2) and the vertical separation distance (d3) are both greater than d1, the center (C1) of the first region is located outside the allowable area (ta). Meanwhile, d1 disclosed in FIG. 2 corresponds to half of the horizontal length and half of the vertical length of the allowable area (ta), and in this case, the preset values ​​for the horizontal length and the vertical length are both d2. Unlike FIG. 2, when the allowable area (ta) is a circular shape, the preset value has a value less than or equal to the length of the radius of the circular shape.

[0054] If the distance is greater than a preset value, the control unit (130) can move the stair-climbing robot (1) so that the distance becomes less than or equal to the preset value (S550). If the distance becomes less than or equal to the preset value due to the movement of the stair-climbing robot (1), the center (C1) of the first area is located within the allowable area (ta), and the control unit (130) can determine that the stair-climbing robot (1) is heading toward the stairs.

[0055] The stair-climbing robot (1) can capture a forward image to generate a second forward image (180B) (S430). Since the stair-climbing robot (1) is facing the stairs by the S420 process, when the S430 process is performed, the second forward image (180B) is generated as an image facing the stairs, as shown in (a) of Fig. 3.

[0056] The stair-climbing robot (1) can remove noise from the second forward image (180B) to generate a third forward image (180C) (S440). According to one embodiment, the S440 process may be a process of extracting a second area (A2) from the second forward image (180B) so that no shape other than stairs is included, and then generating the second area (A2) as a third forward image (180C). The control unit (130) can perform the S440 process to reduce noise or minimize environmental influences.

[0057] The stair-climbing robot (1) can identify the edge of the stairs in the third forward image (180C) (S450). The control unit (130) performs image processing to identify the edge of the stairs in the third forward image (180C) and extract the edge of the stairs (Fig. 3 (c)).

[0058] The S450 process is explained in more detail.

[0059] The control unit (130) can perform gamma correction on the third front image (180C) (S700). Gamma correction is a process of adjusting brightness, and can be performed to accurately extract the step edges by suppressing environmental influences such as illuminance and light.

[0060] The control unit (130) can perform blurring on the third front image (180C) (S710). There are various types of stairs. Each type may have different materials, shapes, patterns, etc. For example, in the case of stairs containing multiple patterns, such as marble stairs, blurring can be performed to reduce the influence of noise, such as marble patterns.

[0061] The control unit (130) can perform channel transformation on the third front image (180C) (S720). Channel transformation refers to the process of converting a three-color channel composed of RGB (red, green, blue) into a two-color channel and converting it into a grayscale such as white and black.

[0062] The control unit (130) can perform binary processing (converting to binary) on the third front image (180C) (S730). Binary processing converts the grayscale image into two colors: black and white. Binary processing enhances the contrast between black and white, enabling effective extraction of the edges of the stairs.

[0063] The control unit (130) can perform filtering on the third front image (180C) (S740). Filtering refers to the process of removing noise to extract the edges of the stairs. The stairs may contain noise such as scratches and foreign substances. By removing external parts of the staircase shape, any elements that interfere with the calculation are eliminated.

[0064] The control unit (130) can identify and extract stair edges (extract stair edges, S750). The S450 process disclosed in FIG. 6 is merely an example, and the S450 process is not limited to the S700 and S750 processes described above. It is also possible to omit some processes or change the order of the processes to identify and extract stair edges.

[0065] The stair-climbing robot (1) calculates the angle between the edge of the stairs and the direction of travel of the stair-climbing robot (1) using the control unit (130) (S460), and aligns the edge of the stairs and the direction of travel of the stair-climbing robot (1) so that they are perpendicular to each other based on the calculated angle between the edge of the stairs and the direction of travel of the stair-climbing robot (1) (S470). This is because, in order to stably climb stairs, the stair-climbing robot (1) must climb perpendicular to the stairs. If it is not perpendicular, the possibility of the stair-climbing robot (1) rolling down or overturning from the stairs while climbing the stairs increases.

[0066] The S470 process is explained in more detail.

[0067] The stair-climbing robot (1) can calculate a movement path (S600). The control unit (130) calculates the movement path of the stair-climbing robot (1) to align the moving direction of the stair-climbing robot (1) perpendicularly with the edge of the stairs. Here, the calculated movement path includes a case where the stair-climbing robot (1) rotates in place. If the angle between the moving direction of the stair-climbing robot (1) and the edge is already close to vertical by the previously performed process S420, the stair-climbing robot (1) can be vertically aligned with the edge of the stairs by simply rotating in place.

[0068] The control unit (130) can control the driving unit (150) so that the stair-climbing robot (1) moves along the calculated movement path (S610). The control unit (130) can determine whether a slip has occurred in the stair-climbing robot (1) while moving along the calculated movement path (S620). If a slip has occurred, the control unit (130) can perform slip compensation control using the second AI engine so that the stair-climbing robot (1) does not deviate from the movement path (S630). The second AI engine may be an AI engine trained to generate a control signal that compensates for the slip of the stair-climbing robot (1) when a slip has occurred in the stair-climbing robot (1).

[0069] Through steps S600 to S630, the direction of movement of the stair-climbing robot (1) can be aligned perpendicular to the edge of the stairs (S640). In this way, when the direction of movement of the stair-climbing robot (1) is perpendicular to the edge of the stairs, the stair-climbing robot (1) can stably navigate the stairs.

[0070] A stair-climbing robot (1) can climb stairs (S480). A control unit (130) controls a driving unit (150), and the stair-climbing robot (1) climbs stairs. While climbing stairs, an inertial measurement device (102) can detect a yaw angle and a pitch angle of the stair-climbing robot (1). Based on the yaw angle and pitch angle detected by the inertial measurement device (102), the control unit (130) can control the posture of the stair-climbing robot (1) so that the direction of movement of the stair-climbing robot (1) and the edge of the stairs remain perpendicular.

[0071] According to one embodiment, the control unit (130) may use a look-up table to control the posture of the stair-climbing robot (1) when running on stairs. In the look-up table, the range of the yaw angle and the range of the pitch angle required for the stair-climbing robot (1) when running on stairs are indexed according to the height of the stairs, the slope of the stairs, the width of the stairs, and the vertical width of the stairs. Here, the horizontal width of the stairs means the width in the direction perpendicular to the vertical width of the stairs, but also the width in the direction parallel to the edge of the stairs. In other words, the look-up table is a kind of reference that presents optimized conditions required for the stair-climbing robot (1) for stable running on stairs. The data stored in the look-up table may vary depending on the type of robot, the specifications of the robot, the type of site, etc.

[0072] According to one embodiment, the control unit (130) can determine the driving state of the stair-climbing robot (1) based on the pitch angle detected by the inertial measurement device (102). The driving state of the stair-climbing robot (1) may include a state before entering the stairs, a state during stair-climbing, and a state after stair-climbing. For example, the control unit (130) can determine that the stair-climbing robot (1) is in a state during stair-climbing when the pitch angle is greater than a specific angle. For example, the control unit (130) can determine that the stair-climbing robot (1) is in a state after stair-climbing when the pitch angle is less than a specific angle.

[0073] In another embodiment, the memory (170) and the control unit (130) may be mounted on a device (not shown). Here, the device may be configured to be mounted on various types of robots that are not equipped with a control function for climbing stairs. The device may be configured to be detachably separated from the robot. Since the device includes a control unit (130) capable of performing the steps S400 to S480 described above, when the device is mounted on a robot, the robot can climb stairs.

[0074] Specifically, even in the case of a robot that is not equipped with a control function for running on stairs, by mounting a device including a control unit (130), the robot can be aligned to face the stairs, aligned perpendicular to the edge of the stairs, and maintained at a constant angle to the edge of the stairs when running on stairs, thereby allowing the robot to run on stairs stably.

[0075] Each component of the device or method according to the present disclosure may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor configured to execute the software functions corresponding to each component.

[0076] Various implementations of the systems and techniques described herein may be implemented as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations of one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special purpose processor or a general purpose processor) coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device. Computer programs (also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."

[0077] A computer-readable recording medium includes any type of recording device that stores data that can be read by a computer system. Such a computer-readable recording medium may be a non-volatile or non-transitory medium such as a ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, storage device, and may further include a transitory medium such as a data transmission medium. Furthermore, the computer-readable recording medium may be distributed across network-connected computer systems, so that computer-readable code can be stored and executed in a distributed manner.

[0078] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations such as changing the order described in the flowchart / timing diagram and executing it or executing one or more of the processes in parallel without departing from the essential characteristics of one embodiment of the present disclosure. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0079] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0080] (Explanation of symbols)

[0081] 1: Stair-climbing robot

[0082] 100: Sensor section

[0083] 101: Camera

[0084] 102: Inertial Measurement Unit

[0085] 130: Control unit

[0086] 150: Drive unit

[0087] 170: Memory

[0088] 180A: First front image

[0089] 180B: Second front image

[0090] 180C: Third front image

[0091]

[0092] CROSS-REFERENCE TO RELATED APPLICATION

[0093] This patent application is filed in Korea on January 11, 2024, under patent application number

[0094] This application claims priority to U.S. Patent No. 10-2024-0004605, the entire contents of which are incorporated herein by reference.

Claims

1. In a control method of a stair-climbing robot, A process of generating a first front image by photographing the front of the stair-climbing robot; A process of identifying the shape of a staircase in the first front image; A process of moving the stair-driving robot toward the stairs based on the shape of the stairs identified in the first forward image so that the stair-driving robot is directed toward the stairs; A process of generating a second front image by photographing the front of the stair-climbing robot; A process of generating a third front image by removing noise from the second front image; A process of identifying the edge of the stairs in the third front image; A process of calculating the angle between the edge of the stairs and the direction of travel of the stair-climbing robot; A process of aligning the stair-driving robot so that the edge of the stairs and the direction of travel of the stair-driving robot are perpendicular to each other based on the angle; and A control method including a process of controlling a driving unit of the stair-climbing robot to allow the stair-climbing robot to climb the stairs.

2. In paragraph 1, The process of identifying the shape of the stairs in the first front image above is as follows: A control method for identifying the shape of stairs in a first front image using a first AI engine learned to identify the shape of stairs in an image.

3. In paragraph 1, The process of moving the stair-driving robot toward the stairs based on the shape of the stairs identified in the first front image is as follows: A process of detecting the exact center of the first front image; A process of displaying a portion of the shape of a staircase identified in the first front image as a first area on the first front image; A process of detecting the exact center of the first region; A process of calculating the distance between the center of the first front image and the center of the first area; and A control method including a process of moving the stair-climbing robot so that the separation distance becomes less than or equal to the preset value when the separation distance is greater than a preset value.

4. In paragraph 1, The process of generating a third front image by removing noise from the second front image is as follows: A control method, which is a process of extracting a second area from the second front image so that no shape other than stairs is included, and then generating the second area as a third front image.

5. In paragraph 1, The process of aligning the stair-driving robot so that the edge of the stairs and the direction of travel of the stair-driving robot are perpendicular to each other based on the angle is as follows. A process of calculating a movement path for moving the above stair-climbing robot; A process of controlling the driving unit of the stair climbing robot so that the stair climbing robot moves along the movement path; and A control method including a process in which the edge of the stairs and the direction of travel of the stair-driving robot are perpendicular to each other.

6. In paragraph 5, If a slip occurs during the process of controlling the driving part of the stair climbing robot so that the stair climbing robot moves along the movement path, Including a process of performing slip compensation control using a second AI engine to prevent the stair-climbing robot from deviating from the movement path; A control method, wherein the second AI engine is an AI engine trained to generate a control signal that compensates for slippage of the stair climbing robot when slippage occurs in the stair climbing robot.

7. In paragraph 1, The process of controlling the driving unit of the stair-climbing robot and causing the stair-climbing robot to climb the stairs is as follows: A process of detecting the yaw angle and pitch angle of the stair climbing robot using an inertial measurement device; and A control method including a process of controlling the posture of the stair climbing robot so that the direction of movement of the stair climbing robot and the edge of the stairs remain perpendicular based on the yaw angle and pitch angle detected by the inertial measurement device.

8. In paragraph 7, The process of controlling the posture of the stair-climbing robot so that the direction of travel of the stair-climbing robot and the edge of the stairs are maintained perpendicular based on the yaw angle and pitch angle detected by the inertial measurement device is as follows. This is a process performed using a look-up table. The above lookup table is, The range of yaw angle and pitch angle required for the stair-climbing robot when climbing stairs are indexed according to the height of the stairs, the slope of the stairs, the width of the stairs, and the height of the stairs. A control method wherein the horizontal width of the above stairs is a width in a direction perpendicular to the vertical width of the stairs, but is a width in a direction parallel to the edge of the stairs.

9. In paragraph 1, A process of detecting the yaw angle and pitch angle of the stair climbing robot using an inertial measurement device; and Further comprising a process of determining the driving status of the stair-climbing robot based on the pitch angle detected by the inertial measurement device, A control method wherein the driving state of the stair-climbing robot includes a state before entering the stairs, a state during stair-climbing, and a state after stair-climbing has ended.

10. In the stair-climbing robot, A camera for filming the front of the stair-climbing robot; A driving unit for moving the above stair-climbing robot; An inertial measurement device for detecting the yaw angle and pitch angle of the stair-climbing robot; and Including the control unit, The above control unit, The front of the stair-climbing robot is photographed to generate a first front image, Identify the shape of the stairs in the first front image above, Based on the shape of the stairs, the stair-driving robot is moved in front of the stairs so that the stair-driving robot is oriented toward the stairs, The front of the stair-climbing robot is photographed to create a second front image, A third front image is generated by removing noise from the second front image, Identify the edge of the stairs in the third front image above, Calculate the angle between the edge of the stairs and the direction of travel of the stairs-climbing robot, Based on the above angle, the stair-driving robot is aligned so that the edge of the stairs and the direction of travel of the stair-driving robot are perpendicular to each other, A stair climbing robot that controls the driving unit so that the stair climbing robot moves up the stairs.

11. In Article 10, The above control unit, When the stair-driving robot is moved in front of the stairs so that the stair-driving robot is oriented toward the stairs based on the shape of the stairs, Detecting the exact center of the first front image, The stairs identified in the first front image are displayed as a first area on the first front image, Detect the exact center of the first region, Calculate the distance between the center of the first front image and the center of the first area, A stair climbing robot that moves the stair climbing robot by controlling the driving unit so that the separation distance becomes less than the preset value when the above-mentioned separation distance is greater than the preset value.

12. In paragraph 10, The above control unit, When the above-mentioned stair-climbing robot controls the driving unit to drive on the stairs, Detecting the yaw angle and pitch angle of the stair climbing robot using an inertial measurement device, A stair climbing robot that controls the posture of the stair climbing robot so that the direction of travel of the stair climbing robot and the edge of the stairs remain perpendicular based on the yaw angle and pitch angle detected by the inertial measurement device.

13. In paragraph 12, The above control unit, When controlling the posture of the stair-climbing robot so that the direction of travel of the stair-climbing robot and the edge of the stairs are kept perpendicular based on the yaw angle and pitch angle detected by the inertial measurement device, a look-up table is used, The above lookup table is, The range of yaw angle and pitch angle required for the stair-climbing robot when climbing stairs are indexed according to the height of the stairs, the slope of the stairs, the width of the stairs, and the height of the stairs. A stair-climbing robot, wherein the horizontal width of the stairs is a width in a direction perpendicular to the vertical width of the stairs, but is a width in a direction parallel to the edge of the stairs.

14. In paragraph 10, A stair-climbing robot, wherein the driving unit further includes a continuous track for moving the robot.

15. A device mounted on a robot including a driving unit, memory; and Including the control unit, The above control unit, The front of the above robot is photographed to generate a first front image, Identify the shape of the stairs in the first front image above, Based on the shape of the stairs, the robot is moved in front of the stairs so that the robot faces the stairs, The front of the robot is photographed to create a second front image, A third front image is generated by removing noise from the second front image, Identify the edge of the stairs in the third front image above, Calculate the angle between the edge of the stairs and the direction of travel of the robot, Based on the above angle, the robot is aligned so that the edge of the stairs and the direction of travel of the robot are perpendicular to each other, A device configured to control the above driving unit to perform a process in which the robot moves on the stairs.

Images