Simultaneous position estimation of camera base for mobilities, and map creation method and system

Abstract

To disclose a simultaneous position estimation for mobilities that can be continued to proceed even when feature points are lost, and a map creation (SLAM) system.SOLUTION: The present invention includes: a camera constituted so as to acquire the front image of a mobility; a movement data sensor constituted so as to detect the movement data of the mobility; and a controller constituted so as to receive the front image of the mobility from the camera and search a feature point from the front image, receive movement data of the mobility from the movement data sensor, and track the searched feature point, and store the state of tracking the feature point. The state includes a first state indicating that the previously tracked first feature point is accurately tracked, a second state indicting that a new second feature point is tracked after the first feature point is lost, and a third state indicating that the first feature point and the second feature point are lost.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method and system for simultaneous localization and mapping based on a camera for mobility. More specifically, even if feature points disappear during simultaneous localization and mapping (SLAM) by object tracking, the present invention relates to a method and system for simultaneous localization and mapping based on a camera for mobility that continues and progresses SLAM by positioning based on a movement data sensor.

Background Art

[0002] Simultaneous Localization And Mapping (SLAM) refers to a technique in which a mobility measures its current position while exploring an unknown environment and simultaneously creates a map of the surrounding environment. Such SLAM is mainly performed based on a lidar. However, lidar-based SLAM is expensive because a lidar must be separately attached to the mobility, and it is rarely applied to mobilities used only indoors rather than for human transportation, and camera-based SLAM has been mainly applied.

[0003] However, in camera-based indoor environment SLAM, it may be difficult to accurately extract feature points from an image detected by the camera due to disturbances such as white walls, featureless planes, and strong light. If the feature points disappear for a long time, SLAM may end. The matters described in this background art section are created to enhance the understanding of the background of the invention and may include matters that are not prior arts already known to those having ordinary knowledge in the field to which this technology belongs.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to provide a mobility camera-based simultaneous localization and mapping method and system that can continue to perform SLAM by position measurement based on a motion data sensor even when feature points disappear during simultaneous localization and mapping (SLAM) by object tracking.

Means for Solving the Problems

[0006] A simultaneous localization and mapping (SLAM) system for mobility according to the present invention includes a camera configured to acquire a front image of the mobility, a motion data sensor configured to detect motion data of the mobility, and a controller configured to receive the front image of the mobility from the camera, search for feature points from the front image, receive the motion data of the mobility from the motion data sensor, track the searched feature points, and store a state of tracking the feature points. The state includes a first state indicating that a first feature point previously tracked is being normally tracked, a second state indicating that a new second feature point is being tracked after the first feature point has disappeared, and a third state indicating that the first feature point and the second feature point have disappeared.

[0007] The controller is further configured to update the position of the mobility based on the motion data of the mobility in the third state.

[0008] The motion data sensor may include an encoder configured to measure information related to the rotation of a drive motor or wheels provided in the mobility and transmit the information to the controller, or an inertial sensor configured to measure information related to the motion state of the mobility including the speed and direction, gravity, and acceleration of the mobility and transmit the information to the controller.

[0009] The controller follows the first feature point in the first state, stores the frame containing the first feature point as the first key frame, temporarily follows the second feature point in the second state after the first state, stores the frame containing the second feature point as the second key frame, determines whether the distance between the first and second key frames is less than a set distance, and in response to the determination that the distance between the first and second key frames is less than the set distance, further configures to input the state to the first state and store the current frame as the first key frame.

[0010] In response to the determination that the distance between the first and second key frames is greater than or equal to the set distance, the controller determines whether the time maintained in the second state is greater than a set time, and in response to the determination that the time maintained in the second state is greater than the set time, further configures to input the state to the first state and follow the second feature point as a new first feature point.

[0011] In response to the determination that the time maintained in the second state is less than the set time, the controller further configures to input the state to the second state and temporarily follow the second feature point.

[0012] In response to the disappearance of the first feature point being followed and the discovery of a new second feature point, the controller further configures to input the state to the second state, store the frame containing the second feature point as the second key frame, and update the mobility position based on the second key frame.

[0013] When the number of discovered feature points is less than or equal to a set number, the controller further configures to input the state to the third state.

[0014] The simultaneous localization and mapping (SLAM) method for mobility according to the present invention is a method for positioning the position of mobility based on feature points or movement data of mobility in one of the following states: a first state indicating that the first feature point that was previously followed is being normally followed, a second state indicating that after the first feature point has disappeared, a new second feature point is being followed, and a third state indicating that the first feature point and the second feature point have disappeared. The method includes, by a controller, receiving a front image of the mobility from a camera in the first state and searching for feature points from the front image; storing, by the controller, a frame including the currently searched first feature point as a first key frame in response to a determination that more feature points than a set number have been searched, and positioning the position of the mobility based on the first feature point; inputting, by the controller, the third state into the state in response to a determination that the number of searched feature points is less than or equal to the set number; and positioning, by the controller, the position of the mobility based on the movement data of the mobility.

[0015] The method may further include, by the controller, receiving a front image of the mobility from the camera in the third state and searching for feature points from the front image; inputting, by the controller, the second state into the state in response to a determination that more feature points than a set number have been searched; and positioning, by the controller, the position of the mobility based on the currently searched second feature point.

[0016] The method may further include, by the controller, receiving a front image of the mobility from the camera in the second state and searching for feature points from the front image; inputting, by the controller, the third state into the state in response to a determination that the number of searched feature points is less than or equal to the set number; and positioning, by the controller, the position of the mobility based on the movement data of the mobility.

[0017] The method may further include: a step in which a controller receives a forward image from a camera in a second state and searches for feature points in the forward image; a step in which, in response to a determination by the controller that more feature points than a set number have been searched, the frame including the currently searched second feature points is stored as a second key frame, and the position of the mobility is temporarily measured based on the second feature points; a step in which the controller determines whether the distance between the first and second key frames is less than a set distance; and a step in which, in response to a determination by the controller that the distance between the first and second key frames is less than the set distance, the state is input to the first state and the current frame is stored as the first key frame.

[0018] The method may further include: a step in which the controller determines whether the time maintained in the second state is greater than a set time in response to a determination that the distance between the first and second key frames is greater than or equal to the set distance; and a step in which, in response to a determination by the controller that the time maintained in the second state is greater than the set time, the state is input to the first state, and the second feature points are regarded as new first feature points and tracked.

[0019] The method may further include a step in which, in response to a determination by the controller that the time maintained in the second state is less than the set time, the state is input to the second state, and the position of the mobility is temporarily measured based on the second feature points.

[0020] A movement data sensor is configured to detect movement data of the mobility, and the movement data sensor may include an encoder configured to measure information regarding the rotation of a drive motor or wheels provided in the mobility and transmit the information to the controller, or an inertial sensor configured to measure information regarding the movement state of the mobility including the speed and direction, gravity, and acceleration of the mobility and transmit the information to the controller.

Advantages of the Invention

[0021] According to the present invention, even if feature points are lost during simultaneous localization and mapping (SLAM) by object tracking, SLAM can continue to progress based on position measurement using a motion data sensor. Therefore, the robustness of SLAM against disturbances such as light can be ensured.

[0022] Also, if the feature points that disappeared during the progress of SLAM based on position measurement using a motion data sensor are detected again, the position of mobility can be corrected based on the previous position and the current position of the feature points. Therefore, SLAM can be accurately performed. In addition, the predicted effects obtained by the embodiments of the present invention will be disclosed in the detailed description of the embodiments of the present invention. That is, various effects predicted by the embodiments of the present invention will be disclosed in the following detailed description.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

[0024] The drawings are not necessarily shown to scale and present somewhat simplified representations of various preferred features that illustrate the basic principles of the present disclosure. For example, certain design features of the present disclosure, including specific dimensions, directions, positions, and shapes, are determined in part by the particular intended application and the usage environment.

Modes for Carrying Out the Invention

[0025] The terms are for illustrative purposes only of specific embodiments and are not intended to limit the present invention. The singular forms also include the plural forms unless the context clearly dictates otherwise. The terms "comprise" and / or "comprising", as used herein, specify the presence of the recited features, integers, steps, operations, components and / or elements, but do not preclude the presence or addition of one or more other features, integers, steps, operations, components, elements and / or groups thereof. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0026] As used herein, the term "mobility" or "mobility of" or other similar terms includes general land mobility including passenger vehicles such as sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, etc., marine mobility including various boats and ships, and air mobility including aircraft, drones, etc., and includes all objects that can move powered by a power source.

[0027] As used herein, the term "mobility" or "mobility of" or other similar terms is understood to include hybrid mobility, electric mobility, plug-in hybrid mobility, hydrogen-powered mobility and other alternative fuel (e.g., fuels derived from resources other than petroleum) mobility. As referred to herein, hybrid mobility includes mobility having two or more power sources, for example, gasoline-powered and electric-powered mobility. Mobility according to embodiments of the present invention includes not only manually driven mobility but also somewhat autonomous and / or automatically driven mobility.

[0028] Additionally, the following method or one or more of these aspects can be executed by at least one controller. The term "controller" refers to a hardware device that includes a memory and a processor. The memory is configured to store program instructions, and the processor is specially programmed to execute the program instructions to perform one or more processes described in more detail below. The controller controls the operation of units, modules, components, devices, or similar things as described herein. Also, the following method can be executed by a device that includes a controller along with one or more other components, as would be recognized by those skilled in the art.

[0029] Also, the controller of the present disclosure can be realized as a non-transitory computer-readable recording medium including executable program instructions executed by a processor. Examples of computer-readable recording media include, but are not limited to, ROM, RAM, compact disc (CD) ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage devices. The computer-readable recording medium can also be distributed across an entire computer network such that the program instructions can be stored and executed in a distributed manner, for example, by a telematics server or a Controller Area Network (CAN).

[0030] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0031] FIG. 1 is a block diagram of a simultaneous localization and mapping system for mobility according to an embodiment of the present invention. As shown in FIG. 1, a simultaneous localization and mapping (SLAM) system for mobility according to an embodiment of the present invention includes an encoder 20, an inertial sensor 30, a camera 40, a controller 50, and a mobility 60.

[0032] The encoder 20 measures information regarding the rotation of the drive motor or wheels provided in the mobility 60. The encoder 20 is connected to the controller 50 and can transmit the measured information regarding the rotation of the drive motor or wheels to the controller 50. The controller 50 can calculate the movement data of the mobility 60, such as the movement speed and / or movement distance of the mobility 60, based on the information regarding the rotation of the drive motor or wheels.

[0033] The inertial sensor 30 measures information regarding the movement status of the mobility 60, including the speed and direction, gravity, and acceleration of the mobility 60. The inertial sensor 30 is connected to the controller 50 and can transmit the measured information regarding the movement status of the mobility 60 to the controller 50. The controller 50 can detect or supplement the movement data of the mobility 60 based on the information regarding the movement status of the mobility 60.

[0034] Here, it is exemplified that both the encoder 20 and the inertial sensor 30 are used as movement data sensors for detecting the movement data of the mobility 60, but only one of the encoder 20 and the inertial sensor 30 may be used as the movement data sensor. Also, the movement data sensor is not limited to the encoder 20 and the inertial sensor 30 and includes various sensors for detecting the movement data of the mobility 60.

[0035] The camera 40 is mounted on the mobility 60 and acquires a front image of the mobility 60 within the measurement range of the camera 40. The camera 40 is connected to the controller 50 and can transmit the acquired image to the controller 50. The controller 50 receives information regarding the rotation of the drive motor or wheels from the encoder 20, receives information regarding the movement status of the mobility 60 from the inertial sensor 30, and receives a front image of the mobility 60 from the camera 40.

[0036] The controller 50 is configured to search for objects (e.g., feature points) in the image by an object search algorithm such as an artificial neural network based on the received forward image. The controller 50 is configured to follow the searched feature points and measure the position of the mobility 60 based on the followed feature points.

[0037] The controller 50 detects the movement data of the mobility 60 based on the information regarding the rotation of the drive motor or wheels received from the encoder 20, or the information regarding the movement status of the mobility 60 received from the inertial sensor 30. If the followed feature point disappears, the controller 50 estimates the position of the mobility 60 based on the movement data of the mobility 60 and measures the absolute position of the mobility 60 by a known position measurement method. If the followed feature point disappears, the controller 50 is configured to perform SLAM based on the absolute position of the mobility 60.

[0038] For such a purpose, the controller 50 is provided with one or more microprocessors, and the one or more microprocessors may be programmed to perform each stage of SLAM according to an embodiment of the present invention.

[0039] The controller 50 is connected to the mobility 60 and generates a path of the mobility 60 or controls the movement of the mobility 60 using a map created by the SLAM method according to an embodiment of the present invention. For example, the controller 50 controls the mobility 60 to follow an object or controls the mobility 60 to avoid an object. FIGS. 2 and 3 are flowcharts of a method for simultaneous position estimation and map creation for mobility according to other embodiments of the present invention.

[0040] As shown in FIG. 2, the method for simultaneous position estimation and map creation according to other embodiments of the present invention is started when the start of the mobility 60 is turned on. For example, the user presses the start button of the mobility 60 or turns on the start of the mobility 60 via the user interface.

[0041] When the activation of Mobility 60 is turned on, the user presses the SLAM button provided on Mobility 60 or triggers SLAM via the user interface (S100). If SLAM is triggered, the controller 50 inputs the first state into the state (S110), and the camera 40 detects the front image of Mobility 60. The controller 50 receives the front image of Mobility 60 from the camera 40 and searches for feature points from the front image (S120). Here, the state means a state of tracking feature points and includes the first state to the third state. The first state means a state of normally tracking the first feature point that was previously tracked, the second state means a state of tracking a new second feature point after the disappearance of the first feature point, and the third state means a state where the first feature point or the second feature point has disappeared. The step S110 means that if SLAM is triggered, the first state is input into the default state.

[0042] If feature points are searched in step S120, the controller 50 determines whether the number of feature points is greater than the set number (N1) and whether the state is the first state (S130). Here, the set number (N1) may be 20, but is not limited thereto. If the number of feature points is greater than the set number (N1) and the state is the first state in step S130, feature point tracking is performed based on the previous frame (S140). In one example, the controller 50 tracks the first feature point searched in the previous frame detected in the previous object search cycle immediately before the current object search cycle. That is, the first feature point searched in the previous frame is tracked, and the feature points searched in the current frame are matched with the first feature point searched in the previous frame. If, by any chance, the previous frame does not exist, that is, if SLAM has just started, the controller 50 performs step S150 without performing step S140.

[0043] If the feature points are tracked based on the previous frame in step S140, the controller 50 inputs the first state into the state and stores the current frame as the first key frame (S150). Here, the first key frame means a frame serving as a reference for tracking the first feature points in the next object search cycle in the first state. That is, all the first feature points being tracked in the first state exist in the first key frame. For example, the first key frame stored in step S150 becomes the previous frame in step S140 in the next object search cycle. After that, the controller 50 measures the position of the mobility 60 based on the tracked first feature points (S160). Here, since the method of measuring the position of the mobility 60 based on the first feature points is well known to those skilled in the art, the description thereof is omitted.

[0044] If the position of the mobility 60 is measured, the controller 50 determines whether SLAM has ended (S170). If SLAM has ended, the method according to the embodiment of the present invention ends. In contrast, if SLAM has not ended, the method proceeds to step S120. When the next object search cycle arrives, the controller 50 searches for feature points from the image acquired by the camera 40.

[0045] On the other hand, if by any chance the number of feature points is less than or equal to the set number (N1) in step S130 or the state is not the first state, the method proceeds to step S200. As shown in FIG. 3, in step S200, the controller 50 determines whether the number of feature points is greater than the set number (N1).

[0046] If it is determined in step S200 that the number of feature points is greater than the set number (N1), the controller 50 determines whether the state is the second state (S210). That is, since the number of feature points is greater than the set number (N1), the controller 50 determines whether the state is the second state or the third state.

[0047] If it is determined in step S210 that the state is the second state, the controller 50 performs feature point tracking based on the previous frame (S220). If the number of feature points is greater than the set number (N1) and the state is the second state, although some of the previously tracked first feature points have disappeared, new second feature points are detected and the second feature points are newly tracked. Therefore, when it is shown that the state is the second state and new second feature points are being tracked in step S210, the controller 50 tracks the second feature points searched in the previous frame detected in the previous object search period immediately before the current object search period. That is, the second feature points searched in the previous frame are tracked, and the feature points searched in the current frame are matched with the second feature points searched in the previous frame.

[0048] If the feature points are tracked based on the previous frame in step S220, the controller 50 stores the current frame as the second key frame (S230). Here, the second key frame means a frame that serves as a reference for tracking the second feature points in the next object search period in the second state. That is, all the second feature points being tracked in the second state exist in the second key frame. For example, the second key frame stored in step S230 becomes the previous frame in step S220 in the next object search period.

[0049] If the second key frame is stored in step S230, the controller 50 determines whether the distance between the first and second key frames is less than the set distance (D1) (S240). If the first key frame where the previously tracked first feature points exist and the second key frame where the currently tracked second feature points exist are close, it is determined that the previously tracked and disappeared first feature points are searched again. Therefore, step S240 is a step of determining whether the currently tracked second feature points are the same as the previously disappeared first feature points. Here, the set distance (D1) can be set to a distance considered appropriate by those skilled in the art.

[0050] If the distance between the first and second key frames is less than the set distance (D1) at stage S240, the method proceeds to stage S150, where the controller 50 inputs the first state into the state and stores the current frame as the first key frame. If the distance between the first and second key frames is less than the set distance (D1), since the lost first feature point has been searched for again, the controller 50 returns the state to the first state and stores the current frame as the first key frame.

[0051] If the distance between the first and second key frames is greater than or equal to the set distance (D1) at stage S240, the controller 50 determines whether the time maintained in the second state is greater than the set time (T1) (S250). If the time maintained in the second state is greater than the set time (T1) at stage S250, it is determined that, although the lost first feature point has not been searched for again, a new second feature point is stably tracked. Thus, the controller 50 inputs the first state into the state (S260), and the method proceeds to stage S120. Therefore, when the next object search cycle arrives, the controller 50 searches for feature points from the image acquired by the camera 40 (S120).

[0052] If the time maintained in the second state is less than or equal to the set time (T1) at stage S250, an effort is made to search for the lost first feature point again, and the second state of tracking the new second feature point is maintained (S270). Thereafter, the method proceeds to stage S120, and when the next object search cycle arrives, the controller 50 searches for feature points from the image acquired by the camera 40.

[0053] If it is determined in step S210 again that the state is not the second state, since the controller 50 has searched for sufficient second feature points for performing feature point tracking although the first feature points being tracked have disappeared, the controller 50 inputs the second state to the state (S290), and starts tracking new second feature points. For this purpose, the controller 50 stores the current frame as the second key frame (S300), and stores the frame serving as a reference for tracking feature points in the next object search cycle in the second state. Thereafter, the controller 50 updates the position of the mobility 60 based on the new second feature points (S310), and the method proceeds to step S160, and the controller 50 measures the position of the mobility 60.

[0054] If again the number of feature points is less than or equal to the set number (N1) in step S200, the controller 50 determines that the first feature points or the second feature points being tracked have disappeared and SLAM based on the feature points cannot be performed, and inputs the third state to the state (S280). Thereafter, the controller 50 updates the position of the mobility 60 based on the movement data of the mobility 60 (S320). For example, the controller 50 detects the movement data of the mobility 60 based on the information regarding the rotation of the drive motor or the wheels received from the encoder 20, or the information regarding the movement state of the mobility 60 received from the inertial sensor 30, and adds the movement data of the mobility 60 to the previous position of the mobility 60 to update the current position of the mobility 60.

[0055] If the position of the mobility 60 is updated, the method proceeds to step S120, and if the next object search cycle arrives, the controller 50 searches for feature points from the image acquired by the camera 40.

[0056] FIG. 4 shows an exemplary scenario of a method for simultaneous position estimation and mapping for mobility according to another embodiment of the present invention.

[0057] In FIG. 4, the controller 50 starts to follow the first feature point 62 at time T and continues to follow the first feature point 62 until time (T + 2). In this case, steps S110 to S170 are repeated. At time (T + 3), the controller 50 loses the first feature point it was following. In this case, since the number of feature points is less than or equal to the set number (N1) at step S130, the method proceeds to step S200. Also, since the number of feature points is less than or equal to the set number (N1) at step S200, the controller 50 inputs the third state into the state (S280) and updates the position of the mobility 60 based on the movement data of the mobility 60.

[0058] At time (T + 4), the controller 50 detects a new second feature point 64 (S120). Although the number of feature points 64 is more than the set number (N1), since the state is the third state, the method proceeds to step S290 through steps S130, S200, and S210. The controller 50 inputs the second state into the state at step S290 and starts to follow the new second feature point 64. Thereby, the controller 50 stores the current frame as the second key frame (S300), updates the position of the mobility 60 based on the new second feature point 64 (S310), and measures the position of the mobility 60 (S160).

[0059] At time (T + 5), the controller 50 detects and follows the second feature point 64 and at the same time detects another first feature point 62 (S120). Although the number of feature points 62 and 64 is more than the set number (N1), since the state is the second state, the method proceeds to step S220 through steps S130, S200, and S210. The controller 50 follows the second feature point 64 based on the previous frame (i.e., the second key frame stored at time (T + 4)) (S220) and stores the current frame as the second key frame (S230). The second key frame includes the currently followed second feature point 64 and the other first feature point 62.

[0060] Thereafter, the controller 50 compares the distance between the first key frame (the first feature point 62 that disappeared after being followed previously) and the second key frame (the second feature point 64 currently being followed and the other first feature points 62) (S240). Since the distance between the first and second key frames is less than the set distance (D1) (that is, since the disappeared first feature point 62 has been found again), the method proceeds to step S150. The controller 50 inputs the first state into the state and stores the current frame as the first key frame. Also, the position of the mobility 60 based on the second feature point 64 and the position of the mobility 60 based on the first feature point 62 can be compared to correct and optimize the position of the mobility 60.

[0061] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to the above embodiments and includes all modifications within the scope recognized as equivalent by those having ordinary knowledge in the technical field to which the present invention pertains from the embodiments of the present invention.

Claims

1. A camera configured to acquire a front image of the mobility, A movement data sensor configured to detect movement data of the mobility, A controller configured to receive a front image of the mobility from the camera, search for feature points from the front image, receive the movement data of the mobility from the movement data sensor, track the searched feature points, and store the state of tracking the feature points, comprising, wherein the state is a first state indicating that a first feature point that was previously tracked is being normally tracked, is a second state indicating that a new second feature point is being tracked after the first feature point has disappeared, is a third state indicating that the first feature point and the second feature point have disappeared, A simultaneous localization and mapping (SLAM) system for mobility, characterized by including the above.

2. The SLAM system for mobility according to claim 1, wherein the controller is further configured to update the position of the mobility based on the movement data of the mobility in the third state.

3. The movement data sensor is an encoder configured to measure information regarding the rotation of a drive motor or wheels provided in the mobility and transmit the information to the controller, or is an inertial sensor configured to measure information regarding the movement state of the mobility including the speed and direction, gravity, and acceleration of the mobility and transmit the information to the controller, The SLAM system for mobility according to claim 1, characterized by including the above.

4. The controller tracks the first feature point in the first state, and stores the frame including the first feature point as the first key frame, after the first state, temporarily tracks the second feature point in the second state, and stores the frame including the second feature point as the second key frame, determines whether the distance between the first and second key frames is less than a set distance, In response to the determination that the distance between the first and second key frames is less than the set distance, the SLAM system for mobility according to claim 1, further configured to input the state to the first state and store the current frame as the first key frame.

5. In response to the determination that the distance between the first and second key frames is greater than or equal to the set distance, the controller determines whether the time maintained in the second state is greater than a set time, In response to a determination that the time maintained in the second state is greater than the set time, the state is input to the first state, and the second feature point is further configured to be regarded as a new first feature point and followed. The SLAM system for mobility according to claim 4, characterized in that.

6. The controller is further configured to input the state to the second state in response to a determination that the time maintained in the second state is less than the set time, and to temporarily follow the second feature point. The SLAM system for mobility according to claim 5, characterized in that.

7. In response to the controller losing the first feature point that was being followed and searching for a new second feature point, the state is input to the second state, the frame containing the second feature point is stored as the second key frame, and the mobility position is further configured to be updated based on the second key frame. The SLAM system for mobility according to claim 1, characterized in that.

8. The controller is further configured to input the state to the third state when the number of searched feature points is less than or equal to the set number. The SLAM system for mobility according to claim 1, characterized in that.

9. Among the first state indicating that the first feature point that was previously followed is being normally followed, the second state indicating that a new second feature point is being followed after the first feature point is lost, and the third state indicating that the first feature point and the second feature point are lost, in a method for simultaneous localization and mapping (SLAM) for mobility that locates the position of mobility based on feature points or movement data of mobility in one of the states, The step of receiving a front image of the mobility from a camera by the controller and searching for feature points from the front image; In response to a determination by the controller that more feature points than the set number have been searched, storing the frame containing the currently searched first feature point as the first key frame, and positioning the position of the mobility based on the first feature point; In response to a determination by the controller that the number of searched feature points is less than or equal to the set number, inputting the third state to the state; The step of positioning the position of the mobility based on the movement data of the mobility by the controller; A SLAM method for mobility, characterized by including.

10. The step of receiving a front image of the mobility from a camera by the controller and searching for feature points from the front image in the third state; In response to a determination by the controller that more feature points than the set number have been searched, entering a second state into the state; Positioning the position of the mobility based on the second feature point currently searched by the controller; The SLAM method for mobility according to claim 9, further comprising the above.

11. Receiving, by the controller, a front image of the mobility from the camera in the second state and searching for feature points from the front image; In response to a determination by the controller that the number of feature points searched is less than or equal to the set number, entering a third state into the state; Positioning the position of the mobility based on the movement data of the mobility by the controller; The SLAM method for mobility according to claim 10, further comprising the above.

12. Receiving, by the controller, a front image of the mobility from the camera in the second state and searching for feature points from the front image; In response to a determination by the controller that more feature points than the set number have been searched, storing the frame including the currently searched second feature point as a second key frame, and temporarily positioning the position of the mobility based on the second feature point; Determining, by the controller, whether the distance between the first and second key frames is less than the set distance; In response to a determination by the controller that the distance between the first and second key frames is less than the set distance, entering the state into the first state and storing the current frame as the first key frame; The SLAM method for mobility according to claim 10, further comprising the above.

13. In response to a determination by the controller that the distance between the first and second key frames is equal to or greater than the set distance, determining whether the time maintained in the second state is greater than the set time; In response to a determination by the controller that the time maintained in the second state is greater than the set time, entering the state into the first state, regarding the second feature point as a new first feature point, and tracking it; The SLAM method for mobility according to claim 12, further comprising the above.

14. The SLAM method for mobility according to claim 13, further comprising, in response to a determination by the controller that the time maintained in the second state is less than the set time, entering the state into the second state and temporarily positioning the position of the mobility based on the second feature point.

15. The mobile data sensor is configured to detect mobility movement data, The mobile data sensor is, an encoder configured to measure information regarding the rotation of a drive motor or wheels provided in the mobility and transmit the information to a controller, or an inertial sensor configured to measure information regarding the movement status of the mobility including the speed and direction, gravity, and acceleration of the mobility and transmit the information to the controller, The SLAM method for mobility according to claim 9, characterized by including the above.

Images