Mobile robot for avoiding rough terrain based on 2D camera and depth camera

Abstract

A mobile robot for rough terrain avoidance according to an embodiment includes a 2D camera that acquires 2D images; Depth camera for acquiring depth images; A preliminary safety area identification unit that distinguishes a danger area and a preliminary safety area using the above 2D image; An actual safety area identification unit that distinguishes between an actual safety area and an actual danger area among the preliminary safety areas using the depth image above; and It may include a driving control unit that controls the moving robot to drive to the actual safe area mentioned above.

Description

Technology Field

[0001] The present invention relates to a mobile robot for rough terrain avoidance based on a 2D camera and a depth camera, and more specifically, to a mobile robot for rough terrain avoidance based on a 2D image using a 2D camera and a depth image using a depth camera to identify a dangerous area and control it to drive to a safe area. Background Technology

[0003] The method of acquiring images of topographic features to identify them, create maps, and store them in a database is generally being practiced.

[0004] In particular, technology that identifies dangerous areas containing obstacles and generates paths to avoid them is also being continuously researched.

[0005] In addition, research is underway on the production of precision maps that identify hazardous areas to support more convenient, stable, and efficient arrival at the destination through autonomous driving.

[0006] However, if the video including the relevant danger area consists only of information captured by a standard 2D camera, the precision may be relatively low, so there is an increasing need to generate a driving path by more accurately identifying the danger area and the safe area. The problem to be solved

[0008] The present invention was derived from the aforementioned necessity and aims to provide a method for generating a driving path by using a 2D camera and a depth camera together to more accurately identify dangerous areas and safe areas. means of solving the problem

[0010] A mobile robot for rough terrain avoidance according to an embodiment includes a 2D camera that acquires 2D images;

[0011] Depth camera for acquiring depth images;

[0012] A preliminary safety area identification unit that distinguishes a danger area and a preliminary safety area using the above 2D image;

[0013] An actual safety area identification unit that distinguishes between an actual safety area and an actual danger area among the preliminary safety areas using the depth image above; and

[0014] It may include a driving control unit that controls the moving robot to drive to the actual safe area mentioned above.

[0015] The above driving control unit is,

[0016] If there are multiple actual safety areas, the actual safety area located relatively closer to the mobile robot among the multiple actual safety areas can be selected as the near-field actual safety area, and the mobile robot can be controlled to drive to the selected near-field actual safety area.

[0017] The above-mentioned preliminary safety zones are multiple, and

[0018] The above actual safety area identification unit is,

[0019] By using the depth image above and referring to the height of each of the plurality of preliminary safety areas, the actual safety area and the actual danger area among the plurality of preliminary safety areas can be distinguished.

[0020] The above actual safety area identification unit is,

[0021] (a) Extract representative points for each of the above multiple preliminary safety zones, and

[0022] (b) Using each of the above representative points, the height of each of the above multiple preliminary safety zones ( ) can be produced.

[0023] The above actual safety area identification unit is,

[0024] The vertical distance between the depth camera and the point corresponding to each representative point ( ) and the vertical distance from the depth camera to the ground ( The above (b) can be performed using the difference between ).

[0025] The above actual safety area identification unit is,

[0026] The straight-line distance between the depth camera and each representative point ( ) and the vertical viewing angle of the pixel corresponding to each of the above representative points ( The vertical distance between the depth camera and the point corresponding to each representative point using ) Calculate ) and,

[0027] The vertical angle of view of the pixel corresponding to each of the above representative points ( ) is the mounting angle of the depth camera for the mobile robot ( ) may have been considered.

[0028] Any of the above multiple preliminary safety areas includes a protruding area protruding relative to the ground, and

[0029] Any representative point among the above representative points represents the center point of the upper part of the above protruding area, and

[0030] The above actual safety area identification unit is,

[0031] The vertical distance between the depth camera and the ground ( The vertical distance between the depth camera and the point corresponding to the arbitrary representative point in ) Subtract the height of the above protruding area ( (2)) can be produced.

[0032] Any of the above multiple preliminary safety areas includes a sunken area based on the ground, and

[0033] Any representative point among the above representative points represents the central point of the lower part of the above-mentioned sunken area, and

[0034] The above actual safety area identification unit is,

[0035] The vertical distance between the depth camera and the point corresponding to the arbitrary representative point ( The vertical distance between the depth camera and the ground in ) The height of the above-mentioned depression area by subtracting ) (1)) can be produced.

[0036] The above actual safe area and the above actual dangerous area are each multiple, and

[0037] The above driving control unit is,

[0038] (a) Determine the location of the plurality of actual safe zones and the location of the plurality of actual dangerous zones, and

[0039] (b) A path can be generated to drive the mobile robot to any of the multiple actual safe areas while avoiding the multiple actual safe areas by using the locations of the multiple actual safe areas and the locations of the multiple actual dangerous areas, and the mobile robot can be controlled to drive along the path.

[0040] The above driving control unit is,

[0041] The above (a) can be performed by determining the location of each of the plurality of actual safe areas using the first representative point of each of the plurality of actual safe areas, and determining the location of each of the plurality of actual dangerous areas using the second representative point of each of the plurality of actual dangerous areas.

[0042] The above driving control unit is,

[0043] Each of the above first representative points is projected onto a reference plane of a model coordinate system based on the position and orientation of the depth camera, and on the reference plane, the position of each of the plurality of actual safe areas is determined using the angle and distance formed by the origin of the model coordinate system and each of the above first representative points, and

[0044] Each of the above second representative points is projected onto the reference plane of the above model coordinate system, and on the reference plane, the location of each of the above multiple actual risk areas can be determined using the angle and distance formed between the origin of the above model coordinate system and each of the above second representative points. Effects of the invention

[0046] According to the present invention, the preliminary safe area is determined primarily by analyzing a 2D image. For example, since there may be ambiguous parts that are difficult to identify as the actual safe area due to factors such as the camera angle of view or the condition of obstacles when using only the 2D image, the actual safe area among the preliminary safe areas can be more clearly identified by secondarily using a depth image together.

[0047] According to the present invention, by first calculating the height value of each preliminary safe area so that the actual safe area and the actual dangerous area can be identified using only the height value, the calculation for such identification can be made simpler and faster. Brief explanation of the drawing

[0049] FIG. 1 shows a side view of a mobile robot (1) for avoiding rough terrain according to an embodiment. FIG. 2 shows a block diagram of a mobile robot (1) for avoiding rough terrain according to an embodiment. FIGS. 3 to 5 are flowcharts showing the operation of a mobile robot (1) for avoiding rough terrain according to an embodiment. FIGS. 6 to 11 are drawings referenced to explain the operation of FIGS. 3 to 5. Specific details for implementing the invention

[0050] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0051] For reference, in each flowchart of the present invention, each step is an example, and the present invention may be applied in the same or similar manner even if each step is changed and / or combined differently.

[0052] FIG. 1 shows a side view of a mobile robot (1) for avoiding rough terrain according to an embodiment.

[0053] FIG. 2 shows a block diagram of a mobile robot (1) for avoiding rough terrain according to an embodiment.

[0054] Referring to FIGS. 1 and 2, a mobile robot (1) for avoiding rough terrain may include a body part (B), a moving part (M), a control part (10), a camera part (20), and a storage part (30).

[0055] The body part (B) is the main body of the mobile robot (1), and the moving part (M) is a means of movement such as a wheel, which is mounted on the lower part of the body part (B) and can be used for the movement of the mobile robot (1).

[0056] The camera unit (20) may include a 2D camera (21) and a 3D camera (22).

[0057] A 2D camera (21) and / or a 3D camera (22) may be placed on the front of the mobile robot (1), specifically on the front of the body part (B) of the mobile robot (1).

[0058] The 2D camera (21) is positioned at the upper part of the front of the mobile robot (1), and the 3D camera (22) can be positioned at a predetermined distance from the 2D camera (21) and positioned lower than the 2D camera (21).

[0059] According to the embodiment, a 2D camera (21) may be positioned at the upper part of the front of the mobile robot (1) to photograph the surrounding environment as a whole, and a 3D camera (22) may be positioned at the lower part of the 2D camera (21) to photograph ambiguous areas of the surrounding environment intensively.

[0060] The 3D camera (22) can be mounted on the front of the mobile robot (1) so as to be tiltable and / or rotatable.

[0061] Mounting angle of the 3D camera (22) for the above-mentioned mobile robot (1) ( ) is the tilting angle of the 3D camera (22) relative to the mobile robot (1), for example, the angle between the front part (center axis) of the mobile robot (1) and the bottom part (center axis) of the 3D camera (22).

[0062] Mounting angle of the 3D camera (22) for the above mobile robot (1) ( It may be stored in advance in the storage unit (30).

[0063] A 2D camera (21) can capture the surrounding environment and obtain a 2D image.

[0064] A 3D camera (22, depth camera) captures the surrounding environment to obtain a 3D image, and a control unit (10) generates 3D coordinates including depth information for an object in the 3D image and can generate a depth map by processing the 3D coordinates.

[0065] The 3D camera (22) includes a lens (not shown) and a pixel array (not shown), and the lens (not shown) can collect at least a portion of the reflected light of light irradiated onto a subject by a light source and send it to the pixel array (not shown). The pixel array (not shown) can receive the light collected by the lens, and specifically, the pixel array (not shown) is composed of a plurality of pixels, and a specific pixel among the plurality of pixels can receive and detect the light collected by the lens (not shown).

[0066] The control unit (10) may include a preliminary safety area identification unit (11) that distinguishes between a dangerous area and a preliminary safety area using a 2D image, an actual safety area identification unit (12) that distinguishes between an actual safety area and an actual dangerous area among the preliminary safety areas using a 3D image (depth image), a driving control unit (13) that controls the mobile robot to drive to the actual safety area, and a calibration unit (14) that pre-calibrates data necessary for the operation of the actual safety area identification unit (12).

[0067] The storage unit (30) can store information about a learning model, a formula, a table such as Table 1, etc. in advance.

[0068] FIGS. 3 to 5 are flowcharts showing the operation of a mobile robot for rough terrain avoidance (1, hereinafter abbreviated as 'mobile robot') according to an embodiment.

[0069] The preliminary safe area identification unit (11) can distinguish between a dangerous area and a preliminary safe area using a 2D image acquired by a 2D camera (21) (s1).

[0070] The preliminary safety area identification unit (11) receives a 2D image from a 2D camera (21) and can identify objects including a danger area and a preliminary safety area using various known object identification algorithms from the 2D image.

[0071] For example, using a learning model trained with neural networks such as CNN and FCNN, it is possible to identify dangerous areas and safe reserve areas from a 2D image, and recognize the location of each of the dangerous areas and safe reserve areas.

[0072] For reference, the learning model may be obtained through training the neural network based on a learning risk region and a learning reserve safe region.

[0073] In the present invention, a danger area is defined as an area where the driving of a mobile robot is judged to be relatively dangerous, such as where an obstacle is located, and a preliminary safe area can be defined as an area where, through 2D image analysis, an obstacle is not located, or where, even if an obstacle is located, the driving of the mobile robot is judged to be relatively safe.

[0074] The actual safe area identification unit (12) can distinguish between the actual safe area and the actual dangerous area among the preliminary safe areas using a 3D image obtained from a 3D camera (22) (s2).

[0075] In the present invention, the preliminary safe area is determined primarily by analyzing a 2D image. For example, since there may be ambiguous parts that are difficult to identify as the actual safe area due to factors such as the camera angle of view or the condition of obstacles when using only the 2D image (e.g., a part that is not the actual safe area being classified as the actual safe area), a 3D image is used secondarily to more clearly identify the actual safe area among the preliminary safe areas.

[0076] To this end, the actual safety area identification unit (12) can extract each representative point of each of the multiple preliminary safety areas from the 3D image (depth image) (s21).

[0077] Specifically, the actual safety area identification unit (12) can extract each representative point using the coordinate values ​​of each preliminary safety area.

[0078] The actual safety area identification unit (12) can extract a representative point corresponding to the center position of the coordinate values ​​of the outermost points (e.g., outermost points of the bounding box) constituting each of the preliminary safety areas.

[0079] The actual safety area identification unit (12) uses each of the above representative points to determine the height of each of the plurality of preliminary safety areas ( ) can be produced (s22).

[0080] Hereinafter, the description will be explained with reference to FIGS. 6 to 8. For reference, FIG. 7 illustrates a case where the preliminary safety area is a recessed area, and FIG. 8 illustrates a case where the preliminary safety area is a protruding area.

[0081] Specifically, the actual safe area identification unit (12) is the vertical distance between the center point of the 3D camera (22) and the points (P1', P2') corresponding to each representative point (P1, P2). ) and the vertical distance from the 3D camera (22) to the ground ( Using the difference between ), the height of each of the above multiple preliminary safety zones ( ) can be produced.

[0082] The actual safe area identification unit (12) is the straight distance between the center point of the 3D camera (22) and each of the representative points (P1, P2). ) and the vertical viewing angle of the pixel corresponding to each of the above representative points (P1, P2) Using ) the vertical distance between the 3D camera (22) and the points (P1', P2') corresponding to each representative point (P1, P2) ( ) can be produced.

[0083] The vertical viewing angle of the pixel corresponding to each of the above representative points (P1, P2) ( ) is the mounting angle of the 3D camera (22) for the mobile robot (1) ( It may be an angle calculated by considering ).

[0084] The actual safety area identification unit (12) is the above ( ) can be defined as the distance of a straight line connecting the center point of the 3D camera (22) and each representative point extracted from S21.

[0085] The actual safe area identification unit (12) identifies the pixel corresponding to each representative point (P1, P2) among the pixels constituting the image plane of the 3D image, and reads out the vertical angle of view mapped to the selected pixel of the pixel array (not shown) of the 3D camera (22) corresponding to the identified pixel, and the vertical angle of view of the pixel corresponding to each representative point (P1, P2) It can be used as ).

[0086] Here, ( ) is the mounting angle of the 3D camera (22) for the mobile robot (1) ( It represents ) and can be verified by reading from the storage unit (30).

[0087] At this time, the actual safe area identification unit (12) refers to the depth map of Fig. 11 stored in the storage unit (30) and the vertical angle of view (previously mapped to each pixel) ) and / or horizontal angle of view( ) can be verified. (The vertical field of view of the pixel corresponding to each of the above representative points (P1, P2) ) is the angle when the 3D camera (22) is mounted on the mobile robot (1), and the vertical angle of view (pre-mapped to the depth map) ) and / or horizontal angle of view( ) can be defined as the angle when the angle of the center point is set to 0 degrees.)

[0088] For reference, a depth map such as that of FIG. 11 sets the center to 0 degrees and the ends to the horizontal and / or vertical angles of view per pixel of the 3D camera (22). Defined as, the horizontal and / or vertical field of view per pixel is It can be set to differ as.

[0089] Here, n is the total vertical angle of view of all pixels ( m represents the number of ) and the total number of horizontal viewing angles of all pixels, and is an identification number that identifies the vertical field of view of a specific pixel, and can be defined as an identification number that identifies the horizontal viewing angle of a specific pixel.

[0090] The actual safety area identification unit (12) refers to Table 1 below and Formula 1 below, ( By generating a right triangle based on the triangulation method using ) as the reference ( ) and the horizontal distance between each representative point (P1, P2) and the points (P1', P2') corresponding to each representative point (P1, P2) ) can be calculated. For reference, information regarding Table 1 below and the formulas below may be stored in advance in the storage unit (30).

[0091] [Formula 1]

[0092]

[0094] Accordingly, the actual safety area identification unit (12) ( The vertical distance from the 3D camera (22) to the ground read from the storage unit (30) and the storage unit ( Using the difference between ), the height of each of the above multiple preliminary safety zones ( ) can be calculated, and the above ( ) can be calculated by rotating the 3D camera (22) so that the central axis of the 3D camera (22) is perpendicular to the ground to extract a representative point from the ground, or it can be calculated in advance through calibration and stored in the storage unit (30), and this will be described later.

[0095] Meanwhile, the actual safe area identification unit (12), in the case of FIG. 7, that is, when the preliminary safe area is a sunken area (e.g., a pit), extracts a representative point (P1) as the central point of the lower part of the sunken area, and the vertical distance between the center point of the 3D camera (22) and the point (P1') corresponding to the representative point (P1) ( The vertical distance between the 3D camera (22) and the ground in ) The height of the above-mentioned depression area by subtracting ) (1)) can be produced.

[0096] And, in the case of FIG. 8, that is, when the preliminary safe area is a protruding area (e.g., a hill), the actual safe area identification unit (12) extracts a representative point (P2) as the center point of the upper part of the protruding area, and the vertical distance between the 3D camera (22) and the ground ( The vertical distance between the center point of the 3D camera (22) and the point (P2') corresponding to the representative point (P2) ( Subtract the height of the above protruding area ( (2)) can be produced.

[0097] And, the actual safety area identification unit (12) has a height of each of the plurality of preliminary safety areas ( By referring to the plurality of preliminary safety areas, the area where the height is less than a predetermined threshold is defined as the actual safety area, and the area where the height is greater than or equal to a predetermined threshold is defined as the actual danger area, thereby distinguishing between the actual safety area and the actual danger area (s23).

[0098] In other words, according to the present invention, by first calculating the height value of each preliminary safe area so that the actual safe area and the actual dangerous area can be identified solely by the height value, the calculation for such identification can be made simpler and faster. That is, by first distinguishing the actual safe area and the actual dangerous area solely by the height value without referencing the location values ​​of the actual safe area and the actual dangerous area in FIG. 9, the speed of the calculation for such distinction can be made faster. Furthermore, in the case of the present invention, by calculating the height value based on triangulation, the calculation is also made simpler, the speed of the calculation is increased, and the accuracy is improved.

[0099] Hereinafter, with reference to FIG. 6, the vertical distance from the 3D camera (22) to the ground ( Describe the calculation method.

[0100] According to an embodiment, in a case where the original terrain was a flat ground but was changed into a danger area due to various factors (natural disasters, changes in natural conditions, etc.), based on the ground Assuming this was calculated first by calibration, the previously calculated in the risk areas of FIGS. 7 to 9 Using It can produce.

[0101] The calibration section (14) is the straight distance between the center point of the 3D camera (22) and the preliminary representative point (PP). ) and the vertical viewing angle of the pixel corresponding to the above preliminary representative point (PP) Using ) the vertical distance from the 3D camera (22) to the ground ( ) can be produced.

[0102] For reference, regarding pixels, vertical angle of view, horizontal angle of view, depth map, etc., the contents of s22 described based on representative points can be applied identically or similarly.

[0103] The calibration unit (14) refers to Table 1 below and Formula 2 below, ( By generating a right triangle based on the triangulation method using ) as the reference ( ) and the horizontal distance between the preliminary representative point (PP) and the point (PP') corresponding to the preliminary representative point (PP) ) can be calculated. For reference, information regarding Table 1 below and the formulas below may be stored in advance in the storage unit (30).

[0104] [Formula 2]

[0105]

[0106] The driving control unit (13) can control the moving robot to drive to an actual safe area (s3).

[0107] The driving control unit (13) can determine the location of each of the plurality of actual safety areas using the first representative point of each of the plurality of actual safety areas (s31).

[0108] Specifically, the driving control unit (13) receives information including each first representative point calculated by the actual safety area identification unit (12) from the actual safety area identification unit (12), projects each first representative point onto a reference plane of a model coordinate system based on the position and orientation of the 3D camera (22), and determines the position of each of the plurality of actual safety areas using the angle and distance formed by the origin of the model coordinate system and each first representative point on the reference plane.

[0109] Referring to FIG. 9, the driving control unit (13) can project each of the first representative points (P) onto a plane corresponding to the front view of a model coordinate system, i.e., a reference plane, in which the posture or angle of the 3D camera (22) is the axis of each three-dimensional coordinate system.

[0110] The driving control unit (13) determines the position of the 3D camera (22) as the origin of the model coordinate system, and on the reference plane, the hypotenuse distance (straight line distance) between the origin and each first representative point (P) ( (1)) received from the actual safety area identification unit (12) Determine by referring to ), and ( second (1) determined), hypotenuse distance between the origin and each first representative point (( (1))) and the horizontal angle between the line extending from the origin toward the direction of a specific axis of the model coordinate system (e.g., y-axis direction) is the horizontal angle of the pixel corresponding to each first representative point (P1). Determine by referring to ) and ( (determined as the angle between them),

[0111] decided (1) Using the angle between them, refer to the right triangle generated based on Table 1 and Equation 3 below and the triangulation method, and the coordinates of each of the first representative points above ( , ) can be calculated, and the location of each of the multiple actual safe areas can be determined using the calculated coordinates.

[0112] [Formula 3]

[0113]

[0114] The driving control unit (13) can determine the location of each of the plurality of actual danger areas using a second representative point of each of the plurality of actual danger areas (s32).

[0115] For reference, the contents described above in s31 may be applied to s32 in the same or similar way.

[0116] Specifically, the driving control unit (13) receives information including each second representative point calculated by the actual safety area identification unit (12) from the actual safety area identification unit (12), projects each second representative point (P') onto a reference plane of a model coordinate system based on the position and orientation of the 3D camera (22), and determines the location of each of the plurality of actual danger areas using the angle and distance formed by the origin of the model coordinate system and each second representative point (P') on the reference plane.

[0117] The driving control unit (13) can project each of the second representative points (P') onto a plane corresponding to the front view of a model coordinate system, i.e., a reference plane, in which the posture or angle of the 3D camera (22) is the axis of each three-dimensional coordinate system.

[0118] (For reference, although the content of the description below is not illustrated, each corresponding configuration used in s31 is defined in the specification by indicating ['(quotation marks)])

[0119] The driving control unit (13) determines the position of the 3D camera (22) as the origin of the model coordinate system, and on the reference plane, the hypotenuse distance (straight line distance) between the origin and each second representative point (P') ( (1)') received from the actual safety area identification unit (12) Decide by referring to '), and ( 'second (1)' determined), hypotenuse distance between the origin and each second representative point (( (1)')) and the horizontal angle between the line extending from the origin toward the direction of a specific axis of the model coordinate system (e.g., y-axis direction) of the pixel corresponding to each second representative point (P') Decide by referring to ')( Determine ' as the angle between them),

[0120] decided (1)' and the angle between them, refer to the right triangle generated based on Table 1 and Equation 3 below and the triangulation method, and the coordinates of each of the above second representative points ( ', It is possible to calculate ') and determine the location of each of the aforementioned multiple actual safe areas using the calculated coordinates.

[0121] [Equation 4]

[0122]

[0123] number sign Explanation of the symbols note 1 Straight line distance between the center point and the preliminary representative point of the 3D camera (22) 2 3D camera mounting angle Mounted angle 3 Horizontal distance between the preliminary representative point (PP) and the point (PP') corresponding to the preliminary representative point (PP). horizontal distance 4 Vertical distance from the 3D camera to the ground 5 Horizontal field of view already mapped to each pixel 6 Vertical field of view mapped to each pixel 7 Horizontal pixel number 8 vertical pixel number 9 Robot reference x-coordinates for each pixel on the horizontal plane 10 Robot-referenced y-coordinates for each pixel on the horizontal plane 11 The straight line distance between the center point of the 3D camera (22) and each of the representative points (P1, P2). 12 Horizontal distance between each representative point (P1, P2) and the points (P1', P2') corresponding to each representative point (P1, P2). 13 The vertical distance between the center point of the 3D camera (22) and the points (P1', P2') corresponding to each representative point (P1, P2). 14 Height of protruding / depressed areas

[0124] According to the present invention, the calculation speed can be made faster by calculating the information using only representative points, rather than calculating the location of the actual safe area and / or actual dangerous area using all points of the actual safe area and / or actual dangerous area.

[0125] The driving control unit (13) can generate a path that allows the mobile robot to drive to any of the multiple actual safe areas while avoiding the multiple actual safe areas by using the locations of the multiple actual safe areas and the locations of the multiple actual dangerous areas, and can control the mobile robot to drive along the path (s33).

[0126] According to an embodiment, the arbitrary actual safety area is a near-field actual safety area located relatively closer to the mobile robot, and the driving control unit (13) can select the near-field actual safety area and generate a path that causes the mobile robot to drive to the near-field actual safety area.

[0127] According to the embodiment, the driving control unit (13) can determine which actual safety area is located closer by referring to the 3D image of the 3D camera (22) or the location of the actual safety area as in FIG. 9.

[0128] According to an embodiment, the driving control unit (13) can determine which actual safe area is located closer by referencing the pixel coordinates of the 2D image, as shown in FIG. 10, based on the 2D image of the 2D camera (21). For example, if two actual safe areas are located such as object1 and object2, it can be determined that object1 is located closer because the absolute value of the y-axis at the bottom of the bounding box is relatively larger.

[0129] According to the embodiment, the driving control unit (13) can generate a main path with a minimum path cost when a starting point (S) and a target point (G: location of a nearby actual safe area) are set. Preferably, the driving control unit (13) can generate a main path using the D* algorithm. The D* algorithm is a search algorithm that minimizes the path cost from the starting point (S) to the target point (G). It can generate an overall path by applying Global Path Planning based on a secured map, and generate a final path by applying Local Path Planning only in necessary areas. That is, when the starting point (S) and the target point (G) are set, the D* algorithm can generate a main path (P) with a minimum path cost while avoiding actual dangerous areas.

[0130] The driving control unit (13) can control the driving so that autonomous driving is performed smoothly along the generated main path. At this time, the driving control unit (13) can control speed, acceleration, etc. based on state information. For example, if the ground is uneven, the driving control unit (13) can control the speed to be lowered so that relatively stable driving is achieved, and if there are no obstacles on the main path being driven, it can accelerate the speed to shorten the travel time.

[0131] According to the present invention, by utilizing the location of the actual safe area and the location of the actual dangerous area provided in real time, respectively, and enabling the mobile robot to drive to the actual safe area while avoiding the actual dangerous area, it is possible to dynamically respond to the external environment that changes in real time.

[0132] The embodiments described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0133] The program instructions recorded on the computer-readable recording medium described above may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0134] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0135] The aspects of this specification may take the form of hardware in whole, software in whole (including firmware, resident software, microcode, etc.), or a computer program product implemented on one or more computer-readable media on which computer-readable program code is implemented.

[0136] The features, structures, effects, etc. described in the embodiments above are included in one embodiment of the present invention and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. exemplified in each embodiment may be combined or modified and implemented in other embodiments by a person skilled in the art to which the embodiments belong. Therefore, details regarding such combinations and modifications should be interpreted as being included within the scope of the present invention.

[0137] Furthermore, although the invention has been described above with reference to embodiments, this is merely illustrative and does not limit the invention. Those skilled in the art will understand that various modifications and applications not exemplified above are possible within the scope of the essential characteristics of the embodiments. For example, each component specifically shown in the embodiments may be modified and implemented. Differences related to such modifications and applications should be interpreted as being included within the scope of the invention as defined in the appended claims.

Claims

Claim 1 A 2D camera for acquiring a 2D image; a depth camera for acquiring a depth image; a preliminary safety area identification unit for distinguishing a danger area and a preliminary safety area using the 2D image; an actual safety area identification unit for distinguishing an actual safety area and an actual danger area among the preliminary safety areas using the depth image; and a driving control unit for controlling a mobile robot to drive to the actual safety area; wherein the preliminary safety areas are plurality of, and the actual safety area identification unit distinguishes the actual safety area and the actual danger area among the plurality of preliminary safety areas by referencing the height of each of the plurality of preliminary safety areas using the depth image, (a) extracts a representative point of each of the plurality of preliminary safety areas, and (b) uses each representative point to the height of each of the plurality of preliminary safety areas ( Calculates the vertical distance between the depth camera and the point corresponding to each representative point ( ) and the vertical distance from the depth camera to the ground ( A mobile robot for rough terrain avoidance that performs the above (b) using the difference between ). Claim 2 A mobile robot for rough terrain avoidance according to claim 1, wherein the driving control unit, when there are multiple actual safety areas, selects an actual safety area located relatively closer to the mobile robot among the multiple actual safety areas as a near-field actual safety area, and controls the mobile robot to drive to the selected near-field actual safety area. Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 In claim 1, the actual safe area identification unit is the straight-line distance between the depth camera and each representative point ( ) and the vertical viewing angle of the pixel corresponding to each of the above representative points ( The vertical distance between the depth camera and the point corresponding to each representative point using ) Calculate ) and the vertical angle of view of the pixel corresponding to each of the above representative points ( ) is the mounting angle of the depth camera for the mobile robot ( A mobile robot for rough terrain avoidance in which ) is taken into consideration. Claim 7 In claim 1, any of the plurality of preliminary safety areas includes a protruding area protruding with respect to the ground, any of the representative points represents the central point of the upper part of the protruding area, and the actual safety area identification unit is the vertical distance between the depth camera and the ground ( The vertical distance between the depth camera and the point corresponding to the arbitrary representative point in ) Subtract the height of the above protruding area ( (2)) A mobile robot for avoiding rough terrain. Claim 8 In claim 1, any of the plurality of preliminary safety areas includes a sunken area with respect to the ground, any of the representative points represents the central point of the lower part of the sunken area, and the actual safety area identification unit is the vertical distance between the depth camera and the point corresponding to the arbitrary representative point ( The vertical distance between the depth camera and the ground in ) The height of the above-mentioned depression area by subtracting ) (1)) A mobile robot for avoiding rough terrain. Claim 9 A mobile robot for avoiding rough terrain according to claim 1, wherein the actual safe area and the actual dangerous area are each a plurality, and the driving control unit (a) determines the location of the plurality of actual safe areas and the location of the plurality of actual dangerous areas, and (b) generates a path that allows the mobile robot to drive to any of the plurality of actual safe areas while avoiding the plurality of actual dangerous areas using the location of the plurality of actual safe areas and the location of the plurality of actual dangerous areas, and controls the mobile robot to drive according to the path. Claim 10 A mobile robot for rough terrain avoidance, wherein the driving control unit determines the location of each of the plurality of actual safe areas using a first representative point of each of the plurality of actual safe areas, and determines the location of each of the plurality of actual dangerous areas using a second representative point of each of the plurality of actual dangerous areas, thereby performing (a). Claim 11 A mobile robot for rough terrain avoidance according to claim 10, wherein the driving control unit projects each of the first representative points onto a reference plane of a model coordinate system based on the position and attitude of the depth camera, determines the location of each of the plurality of actual safe areas using the angle and distance formed between the origin of the model coordinate system and each of the first representative points on the reference plane, projects each of the second representative points onto the reference plane of the model coordinate system, and determines the location of each of the plurality of actual dangerous areas using the angle and distance formed between the origin of the model coordinate system and each of the second representative points on the reference plane.

Images