System and method for mobility object tracking including lidar and camera

Abstract

To provide a system and a method for mobility object tracking which allows tracking of an object to be performed in real time, on the basis of LIDAR data and mobility movement data, when searching for an object through an artificial nerve net.SOLUTION: A mobility object tracking method includes the steps of: acquiring an image using a camera; starting an object search with respect to the total image through an object search algorithm using a controller; determining whether or not the object search using a controller has been completed; predicting an object search range based on movement data of a mobility and LIDAR data using the controller, when the object search is determined to be not completed; and conducting an object search for the object search range predicted through the object search algorithm using the controller.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an object tracking system and method for mobility, and more particularly to an object tracking system and method for mobility having a LIDAR and a camera that predicts an object search range based on LIDAR data and mobility movement data during object search through an artificial neural network and tracks an object in real time. [Background technology]

[0002] The indoor autonomous mobility can track a specific object. To track a specific object, the indoor autonomous mobility mainly searches for the object through an artificial neural network. More specifically, the indoor autonomous mobility acquires an image using a camera, searches for an object through an artificial neural network in the acquired image, and when the object search is completed, generates a bounding box including the searched object and locates the position of the searched object (e.g., the position of a feature point).

[0003] However, since the maximum speed of searching for objects through an artificial neural network is 30 FPS, while the control period of sensors such as inertial measurement units (IMUs) in mobility is 10 ms (= 100 FPS), it was difficult to integrate sensor data into object searching logic.In addition, since embedded systems have low computing capabilities due to the lack of a graphic processing unit (GPU), it is difficult to search for objects through an artificial neural network in real time.

[0004] Furthermore, when object tracking relies on a remote system, if communication between the remote system and the mobility is interrupted for a long period of time or if the mobility moves significantly differently from the planned route, there is difficulty in tracking the object even when communication with the remote system is restored.

[0005] The matters described in this Background Art section are prepared for the purpose of enhancing understanding of the background of the invention, and may include matters that are not conventional art already known to those having ordinary skill in the art to which this technology pertains. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2020-529686 Summary of the Invention [Problem to be solved by the invention]

[0007] The object of the present invention is to provide an object tracking system and method for mobility equipped with a LIDAR and a camera that predicts an object search range based on LIDAR data and mobility movement data during object search through an artificial neural network and tracks objects in real time. [Means for solving the problem]

[0008] The object tracking system for mobility according to the present invention includes a lidar attached to the mobility and configured to detect lidar data, a movement data sensor attached to the mobility and configured to detect mobility movement data, a camera attached to the mobility and acquiring an image in front of the mobility, and a controller configured to receive lidar data from the lidar, receive mobility movement data from the movement data sensor, receive images from the camera, and search for objects from the received images through an object search algorithm, wherein before completing object search for the entire image through the object search algorithm, the controller is configured to predict a position of a first bounding box based on a position of the bounding box and a previous speed of the mobility, predict a position of a second bounding box based on the position of the bounding box and reference lidar data for a reference lidar point on the image, predict a position of a final bounding box based on the positions of the first and second bounding boxes, and perform object search for the final bounding box through the object search algorithm.

[0009] In response to performing an object search on the final bounding box, upon determining that an object is present within the final bounding box, the controller may be further configured to locate a position of the object.

[0010] In response to performing an object search on the final bounding box, if it is determined that an object does not exist within the final bounding box, the controller may be further configured to move the final bounding box by a set number of pixels and perform an object search again on the moved final bounding box.

[0011] If no object is present within the final bounding box even after the final bounding box has been moved a set number of times, the controller may perform an object search on the entire image through an object search algorithm.

[0012] The controller may be configured to convert variations in the reference lidar data into variations in pixel data and predict a position of the second bounding box based on the variations in the pixel data.

[0013] The controller may be configured to filter the first bounding box and the second bounding box to predict a position of a final bounding box.

[0014] The reference lidar point may be at least one of the two end points on the image.

[0015] If the previous speed of the mobility is not stored or a bounding box does not exist, the controller may perform an object search on the entire image through an object search algorithm.

[0016] The object tracking method for mobility according to the present invention is characterized by including the steps of acquiring an image by a camera, starting an object search for the entire image through an object search algorithm by a controller, determining whether the object search is completed by the controller, and if it is determined that the object search is not completed, predicting an object search range based on movement data and LIDAR data of the mobility by the controller, and performing object search for the object search range predicted through the object search algorithm by the controller.

[0017] The step of predicting an object search range based on the movement data and lidar data of the mobility includes the steps of predicting a position of a first bounding box based on a position of the bounding box and a previous speed of the mobility, predicting a position of a second bounding box based on the position of the bounding box and reference lidar data for a reference lidar point on the image, and predicting a position of a final bounding box as the object search range based on the positions of the first bounding box and the second bounding box.

[0018] If it is determined that an object exists within the final bounding box, the object tracking method may further include determining a position of the object by a controller.

[0019] If it is determined that an object does not exist within the final bounding box, the object tracking method may further include the steps of moving the final bounding box by pixels set by the controller, and re-performing object search on the moved final bounding box through an object search algorithm.

[0020] If no object is present within the final bounding box even after moving the final bounding box a set number of times and repeating the object search, the object tracking method may further include a step of performing object search on the entire image through an object search algorithm by a controller.

[0021] Predicting the position of the second bounding box based on the position of the bounding box and the reference lidar data relative to the reference lidar point on the image may include converting a change in the reference lidar data into a change in pixel data, and predicting the position of the second bounding box based on the position of the bounding box and the change in pixel data.

[0022] The step of predicting the position of a final bounding box as an object search range based on the positions of the first bounding box and the second bounding box may further include filtering the first bounding box and the second bounding box.

[0023] The reference lidar point may be at least one of the two end points on the image.

[0024] If the previous speed of the mobility is not stored or a bounding box does not exist, the object tracking method may further include performing an object search on the entire image through an object search algorithm by the controller. Effect of the Invention

[0025] According to the present invention, during object search through an artificial neural network, an object search range is predicted based on LIDAR data and mobility movement data, and the object can be tracked in real time. In addition, real-time object tracking is possible through efficient calculation without improving the system specifications. Furthermore, since the object search range predicted based on LIDAR data and the object search range predicted based on mobility movement data are filtered to determine the final object search range, the object position can be accurately tracked in real time with an efficient amount of calculation. In addition, the effects obtained or predicted by the embodiments of the present invention will be directly or implicitly disclosed in the detailed description of the embodiments of the present invention. In other words, various effects predicted by the embodiments of the present invention will be disclosed in the detailed description to be described later. [Brief description of the drawings]

[0026] [Figure 1] 1 is a block diagram of an object tracking system for mobility according to an embodiment of the present invention; [Diagram 2] 11 is a flowchart of an object tracking method for mobility according to another embodiment of the present invention. [Diagram 3] 11 is a flowchart of an object tracking method for mobility according to another embodiment of the present invention. [Figure 4] FIG. 1 is a schematic diagram illustrating the relationship between LIDAR data and pixel data. [Diagram 5] 1 is a schematic diagram showing predicting an object search range based on LIDAR data; [Figure 6] 1 is a schematic diagram showing an object search range predicted based on LIDAR data and an object search range predicted based on mobility movement data; [Figure 7] 11 is a schematic diagram showing a final object search range obtained by filtering an object search range predicted based on LIDAR data and an object search range predicted based on mobility movement data; FIG.

[0027] It should be understood that the above-referenced drawings are not necessarily drawn to scale, but rather present somewhat simplified representations of various preferred features illustrating the underlying principles of the present disclosure. For example, specific design features of the present disclosure, including specific dimensions, orientations, locations, and shapes, are determined in part by the particular intended application and environment of use. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting of the present invention. As used herein, the singular forms are intended to include the plural forms unless the context clearly dictates otherwise. The terms "comprises" and / or "comprising", as used herein, are to be understood as specifying the presence of stated features, integers, steps, operations, elements, and / or components, but not excluding the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any one or all combinations of the associated listed items.

[0029] As used herein, "mobility" or "mobility" or other similar terms include general land mobility, including passenger cars, including sports utility vehicles (SUVs), buses, trucks, various commercial vehicles, etc., marine mobility, including various boats and ships, and aerial mobility, including aircraft, drones, etc., and include any object that can be moved by power from a power source. In addition, as used herein, "mobility" or "mobility" or other similar terms include hybrid mobility, electric mobility, plug-in hybrid mobility, hydrogen-powered mobility, and other alternative fuel (e.g., fuel derived from resources other than petroleum) mobility. As referred to herein, hybrid mobility includes mobility with two or more power sources, for example, gasoline-powered and electric-powered mobility. Mobility according to embodiments of the present invention includes not only manually driven mobility, but also mobility that is driven somewhat autonomously and / or automatically.

[0030] Additionally, it is understood that one or more of the methods or aspects thereof described below can be performed by at least one or more controllers. The term "controller" refers to a hardware device including a memory and a processor. The memory is configured to store program instructions, and the processor is specifically programmed to execute the program instructions to perform one or more processes, as described in more detail below. The controller controls the operation of a unit, module, component, device, or the like, as described herein. Additionally, the methods described below can be performed by a device that includes a controller along with one or more other components, as will be appreciated by those skilled in the art.

[0031] The controller of the present disclosure may also be embodied as a non-transitory computer readable recording medium containing executable program instructions executed by a processor. Examples of computer readable recording media include, but are not limited to, ROM, RAM, compact disk (CD) ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer readable recording media may also be distributed throughout a computer network, with the program instructions stored and executed in a distributed manner, such as a telematics server or a controller area network (CAN).

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0033] 1 is a block diagram of an object tracking system for mobility according to an embodiment of the present invention. As shown in FIG. 1, the object tracking system for mobility according to an embodiment of the present invention includes a lidar 10, an encoder 20, an inertial sensor 30, a camera 40, a controller 50, and a mobility 60.

[0034] The LIDAR 10 is attached to the mobility 60, and irradiates a laser pulse ahead of the mobility 60, and measures the time it takes for the laser pulse reflected by an object within the measurement range of the LIDAR 10 to return, thereby detecting information about the object, such as the distance from the LIDAR 10 to the object, the object's direction, speed, temperature, material distribution, and concentration characteristics. Here, the object may be another mobility, person, thing, etc. that exists outside the mobility 60 to which the LIDAR sensor 10 is attached, but the present invention is not particularly limited to the type of object. The LIDAR 10 is connected to the controller 50, detects LIDAR data (e.g., data of multiple LIDAR points) within the measurement range, and transmits the LIDAR data to the controller 50.

[0035] The encoder 20 measures information related to the rotation of a drive motor or wheels provided in the mobility 60. The encoder 20 is connected to the controller 50 and transmits the measured information related to the rotation of the drive motor or wheels to the controller 50. The controller 50 calculates mobility movement data such as the moving speed and / or moving distance of the mobility 60 based on the information related to the rotation of the drive motor or wheels.

[0036] The inertial sensor 30 measures information related to the movement status of the mobility 60, including the speed and direction, gravity, and acceleration of the mobility 60. The inertial sensor 30 is connected to the controller 50 and transmits information related to the measured movement status of the mobility 60 to the controller 50. The controller 50 detects or supplements mobility movement data based on the information related to the movement status of the mobility 60.

[0037] Here, both the encoder 20 and the inertial sensor 30 are used as a movement data sensor for detecting the movement data of the mobility 60, but only one of the encoder 20 and the inertial sensor 30 may be used as a movement data sensor. In addition, the movement data sensor is not limited to the encoder 20 and the inertial sensor 30, and may be various sensors that detect the movement data of the mobility 60.

[0038] The camera 40 is attached to the mobility 60 and captures an image in front of the mobility 60 within the measurement range of the camera 40. The camera 40 is connected to the controller 50 and transmits the captured image to the controller 50. The image is composed of pixel data including a plurality of pixels. Here, the size of the pixel data of the image may be (N*M), where N and M are natural numbers.

[0039] The controller 50 receives lidar data from the lidar 10, information regarding the rotation of the drive motor or wheels from the encoder 20, information regarding the movement status of the mobility 60 from the inertial sensor 30, and images of the front of the mobility 60 from the camera 40.

[0040] The controller 50 is configured to search for an object (e.g., a feature point) in the image through an object search algorithm such as an artificial neural network based on the received image, generate a bounding box when an object in the image is found, and locate the position of the object (e.g., a feature point) in the bounding box. The controller 50 is configured to detect mobility movement data based on information on the rotation of the drive motor or wheels received from the encoder 20 or information on the movement status of the mobility 60 received from the inertial sensor 30, and predict the position of a first bounding box for searching for an object based on the mobility movement data.

[0041] The controller 50 is also configured to predict the position of a second bounding box for searching for an object based on the LIDAR data received from the LIDAR 10. The controller 50 is further configured to predict the position of a final bounding box by filtering the positions of the first bounding box and the second bounding box, determine whether an object (e.g., a feature point) is present within the predicted final bounding box, and, if an object is present within the final bounding box, locate the position of the object.

[0042] For this purpose, the controller 50 may include one or more microprocessors, which may be programmed to perform each step of the object tracking method according to an embodiment of the present invention. The controller 50 is connected to the mobility 60 and controls the movement of the mobility 60 based on the object tracking result. For example, the controller 50 controls the mobility 60 to follow the object or to avoid the object.

[0043] Figures 2 and 3 are flowcharts of an object tracking method for mobility according to another embodiment of the present invention, Figure 4 is a diagram showing a schematic relationship between lidar data and pixel data, Figure 5 is a schematic diagram showing predicting an object search range based on lidar data, Figure 6 is a schematic diagram showing an object search range predicted based on lidar data and an object search range predicted based on mobility movement data, and Figure 7 is a schematic diagram showing a final object search range obtained by filtering the object search range predicted based on lidar data and the object search range predicted based on mobility movement data.

[0044] 4 and 5, (x, y) indicates lidar data detected by lidar 10, and (u, v) indicates pixel data detected by camera 40. In Figures 5 to 7, reference numeral "70" indicates an image detected by camera 40, and reference numeral "72" exemplarily indicates a lidar pointer on image 70.

[0045] 2, the object tracking method for mobility according to another embodiment of the present invention starts by acquiring an image in front of the mobility 60 through the camera 40 (S100). For example, the camera 40 acquires an image in front of the mobility 60 at set intervals and transmits the acquired image to the controller 50. The image is composed of pixel data of size (N*M).

[0046] The controller 50 starts searching for an object in the image (S110). For example, the controller 50 starts searching for feature points of an object present in the image through an artificial neural network, etc. Then, the controller 50 determines whether the object search through the artificial neural network, etc. is completed (S120). If it is determined that the object search is completed in step S120, the controller 50 generates a bounding box including the searched object (e.g., feature point) (S130) and locates the position of the object (e.g., feature point) (S140). Here, the size of the bounding box may be (n*m), where n is a natural number less than N and m is a natural number less than M. However, the size of the bounding box is not limited thereto.

[0047] If it is determined in step S120 that the object search is not complete, the controller 50 determines whether there is a previous stored speed of the mobility 60 and whether there is a stored bounding box, as shown in Fig. 3 (S150). That is, the controller 50 determines whether there is a speed of the mobility 60 detected at a previous time point (i.e., a time point at which the previous position was measured) and stored in memory, and whether there is a bounding box generated at a previous time point and whose position is stored in memory. If there is no previous speed of the mobility 60 stored in memory or no bounding box in step S150, the object tracking method for mobility according to another embodiment of the present invention ends, and the controller 50 performs object search on the entire image through an object search algorithm such as an artificial neural network.

[0048] In step S150, if the previous speed and bounding box of the mobility 60 are stored in memory, the controller 50 predicts the position of the first bounding box 76 based on the previous speed of the mobility 60 (S160). As shown in Figures 5 and 6, the controller 50 predicts the movement amount of the mobility 60 based on the previous speed of the mobility 60 and the time interval between the previous time point and the current time point, and moves the bounding box 74 stored in memory based on the predicted movement amount of the mobility 60 to predict the position of the first bounding box 76.

[0049] Controller 50 also acquires lidar data (x, y) from lidar 10, acquires an image from camera 40, acquires pixel data (u, v) (S170), and calculates a transformation matrix for converting the lidar data into pixel data based on the acquired lidar data (x, y) and pixel data (u, v) (S180). Because lidar 10 and camera 40 are fixedly attached to mobility 60, the transformation matrix for converting the lidar data into pixel data is generally constant.

[0050] Using this fact, a transformation matrix for converting the LIDAR data into pixel data is calculated. For example, as shown in Fig. 4, LIDAR 10 detects the vertices of an object from the LIDAR data (x,y), and camera 40 also detects the same vertices of the same object from the pixel data (u,v). Controller 50 calculates a transformation matrix for converting the LIDAR data (x,y) into pixel data (u,v) based on the LIDAR data (x,y) and the pixel data (u,v).

[0051] Meanwhile, steps S170 and S180 may be optional steps. That is, once a transformation matrix for converting LIDAR data into pixel data is calculated, the transformation matrix is ​​stored in memory. Also, the transformation matrix can be calculated once when LIDAR 10 and camera 40 are provided in mobility 60, and is stored in advance in memory. Thereafter, the controller reads the transformation matrix stored in memory instead of steps S170 and S180.

[0052] Thereafter, the controller 50 locates the reference lidar data on the image (S190). That is, the controller 50 locates a reference lidar point that is always located at a fixed position on the image 70, and locates the reference lidar data. Here, the reference lidar point may be at least one of the two end points on the image. A plurality of lidar points may be located on the same horizontal plane, and among the plurality of lidar points, a lidar point that is located at at least one of the two end points on the image always has the same relative position on the image. Therefore, at least one of the two end points on the image is designated as a reference lidar point, and the reference lidar point is located to obtain the reference lidar data.

[0053] Thereafter, the controller 50 determines whether there is previous reference LIDAR data stored in the memory (S200). That is, the controller 50 determines whether reference LIDAR data for the same reference LIDAR point was previously measured and stored. If it is determined in step S200 that there is no previous reference LIDAR data, the controller 50 stores the reference LIDAR data measured at the current time (S210), and the object tracking method for mobility according to another embodiment of the present invention is terminated, and the controller 50 searches for an object through an object search algorithm such as an artificial neural network for the entire image.

[0054] If it is determined in step S200 that previous reference lidar data exists, the controller 50 stores the reference lidar data measured at the current time, and calculates the amount of change in the reference lidar data based on the reference lidar data measured at the current time and the previous reference lidar data stored in the memory (S220). In addition, the controller 50 calculates the amount of change in pixel data corresponding to the amount of change in the reference lidar data using a transformation matrix (S230). As described above, since the lidar 10 and the camera 40 are fixedly attached to the mobility 60, the amount of change in pixel data corresponds to the amount of change in the position of the mobility 60. Therefore, the controller 50 calculates the amount of change in pixel data to predict the amount of movement of the mobility 60.

[0055] Thereafter, the controller 50 predicts the position of the second bounding box 78 based on the amount of change in the pixel data (S250), as shown in Fig. 5. That is, the controller 50 predicts the position of the second bounding box 78 by moving the bounding box 74 stored in the memory according to the amount of movement of the mobility 60 predicted based on the reference lidar data.

[0056] 6 and 7, the controller 50 filters a first bounding box 76 predicted based on the movement data of the mobility 60 and a second bounding box 78 predicted based on LIDAR data (e.g., reference LIDAR data) to predict the position of a final bounding box 80 (S250). Here, various filters such as a Kalman filter and a low-pass filter may be used as a filter for filtering the position of the bounding box. When the position of the final bounding box 80 is predicted in step S250, the controller 50 determines whether an object exists within the predicted final bounding box 80 (S260). That is, the controller 50 searches for whether a feature point of an object exists within the final bounding box 80 whose size has been reduced to (n*m) through an object search algorithm such as an artificial neural network.

[0057] If an object does not exist within the final bounding box 80 in step S260, the controller 50 moves the final bounding box 80 by the set number of pixels and again determines whether an object exists within the moved final bounding box 80 (S260). If an object does not exist within the final bounding box 80 even after moving the final bounding box 80 the set number of times, the method for tracking an object for mobility according to another embodiment of the present invention is terminated, and the controller 50 performs an object search for the entire image through an artificial neural network, etc.

[0058] If an object is present within the final bounding box 80 in step S260, the controller 50 determines the position of the object (S280). To this end, the controller 50 may continuously perform object tracking using an object search range predicted based on the mobility movement data and an object search range predicted based on the LIDAR data before completing the object search through an artificial neural network, etc. Thereafter, the controller 50 may terminate the object tracking method for mobility according to another embodiment of the present invention.

[0059] Meanwhile, as an optional step, the controller 50 may store the movement data of the mobility 60 detected at the current point in time, particularly the speed of the mobility 60 (S290), and use it to predict the position of the first bounding box 76 at a future point in time. In this way, in the embodiment of the present invention, instead of performing the object search algorithm on the entire image of size N*M detected by the camera 40, the object search algorithm is performed only on the final bounding box of size n*m, which is the reduced size, thereby accurately tracking the position of the object in real time with an efficient amount of calculation.

[0060] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and includes any modifications that can be easily modified from the embodiments of the present invention by a person having ordinary knowledge in the technical field to which the invention pertains and that are deemed to be equivalent.

Claims

1. In the mobility object tracking system, a lidar attached to the mobility and configured to detect lidar data; A movement data sensor attached to the mobility and configured to detect mobility movement data; A camera attached to the mobility and configured to capture an image in front of the mobility; a controller configured to receive lidar data from the lidar, receive mobility movement data from the movement data sensor, receive images from the camera, and search for an object in the received images through an object search algorithm; Including, The controller is configured to, before completing object search for the entire image through an object search algorithm, predict a position of a first bounding box based on a position of the bounding box and a previous speed of the mobility, predict a position of a second bounding box based on the position of the bounding box and reference lidar data for a reference lidar point on the image, predict a position of a final bounding box based on the positions of the first and second bounding boxes, and perform object search for the final bounding box through the object search algorithm.

2. 2. The object tracking system of claim 1, wherein the controller is further configured to locate a position of the object when it is determined that an object exists within the final bounding box in response to performing an object search for the final bounding box.

3. 3. The object tracking system for mobility of claim 2, wherein when it is determined that an object does not exist within the final bounding box in response to performing an object search on the final bounding box, the controller is further configured to move the final bounding box by a set number of pixels and perform object search again on the moved final bounding box.

4. 4. The object tracking system for mobility of claim 3, wherein if an object is not present within the final bounding box even after the final bounding box has been moved a set number of times, the controller performs object search for the entire image through an object search algorithm.

5. 2. The object tracking system of claim 1, wherein the controller is configured to convert a change in the reference lidar data into a change in pixel data, and predict a position of a second bounding box based on the change in the pixel data.

6. The system of claim 1 , wherein the controller is configured to predict a position of a final bounding box by filtering the first bounding box and the second bounding box.

7. The object tracking system for mobility of claim 1 , wherein the reference rider point is at least one of both end points on an image.

8. The object tracking system of claim 1, wherein if the previous speed of the mobility is not stored or a bounding box does not exist, the controller performs object search on the entire image through an object search algorithm.

9. acquiring an image by a camera; initiating an object search for the entire image through an object search algorithm by a controller; determining whether the object search is completed by the controller; predicting an object search range based on the movement data and the lidar data of the mobility by the controller when it is determined that the object search is not completed; performing an object search for an object search range predicted through an object search algorithm by the controller; The object tracking method for mobility, comprising:

10. The step of predicting an object search range based on the mobility movement data and the lidar data includes: predicting a position of a first bounding box based on the position of the bounding box and a previous speed of the mobility; predicting a location of a second bounding box based on the location of the bounding box and the reference lidar data relative to a reference lidar point on the image; predicting a position of a final bounding box as an object search range based on the positions of the first bounding box and the second bounding box; The method of claim 9, further comprising:

11. The method of claim 10, further comprising: determining a position of the object by a controller when it is determined that the object exists within the final bounding box.

12. if it is determined that an object does not exist within the final bounding box, moving the final bounding box by a number of pixels set by a controller; performing object search again on the moved final bounding box through an object search algorithm; The method of claim 11, further comprising:

13. The object tracking method for mobility of claim 12, further comprising the step of performing object search on the entire image through an object search algorithm by a controller if no object is present within the final bounding box even after moving the final bounding box a set number of times and repeating the object search.

14. Predicting a location of a second bounding box based on the location of the bounding box and reference lidar data for a reference lidar point on the image, comprising: converting the change in the reference LIDAR data into a change in pixel data; predicting a position of a second bounding box based on the position of the bounding box and the pixel data change; The method of claim 10, further comprising:

15. predicting a position of a final bounding box as an object search range based on a position of the first bounding box and a position of the second bounding box, filtering the first bounding box and the second bounding box; The method of claim 10, further comprising:

16. The method of claim 10, wherein the reference rider point is at least one of both end points on an image.

17. The object tracking method for mobility of claim 10, further comprising the step of performing object search on the entire image through an object search algorithm by the controller if the previous speed of the mobility is not stored or a bounding box does not exist.

Images