Object tracking system and method capable of stable position estimation of objects that are unstable to radar sensor shape recognition

Abstract

A system for recognizing an object through artificial intelligence (AI)-based processing and at least one autonomous driving sensor including at least one radar sensor, and a method thereof. Operations include generating at least one cluster having position information and a velocity vector, generating cluster information corresponding to the at least one cluster by clustering a plurality of point data obtained from the at least one radar sensor, correcting lateral position information among the position information by applying a moving average to the lateral position information, and updating trajectory information including heading information of a trajectory by recalculating a lateral velocity of the velocity vector based on the position information including the corrected lateral position information.

Description

Technical Field

[0001] This disclosure relates to a system and method for tracking objects using autonomous driving sensors based on artificial intelligence (AI). Background Technology

[0002] As autonomous vehicles are developed and commercialized, the number of cases using various sensors and artificial intelligence (AI) technologies to support autonomous driving functions is increasing. For example, research is currently underway on what objects may be in front of a vehicle, the distance between those objects and the vehicle, and what algorithms the vehicle should use to handle specific situations to ensure safety.

[0003] As a result, vehicle sensor technology is becoming increasingly advanced, and various high-performance sensors have been installed on vehicles, such as LiDAR (Light Detection and Ranging) which uses laser beams to identify the surrounding environment, Radar (Radio Detection and Ranging) which uses radio waves, ultrasonic sensors, fisheye cameras that can capture 360-degree images, multifocal lenses, and Global Positioning System (GPS).

[0004] By aggregating measurement results from multiple sensors, a so-called super-sensor vehicle can be realized. In autonomous driving (or driverless driving), the concept of a super-sensor refers to a technology aimed at more accurately identifying the surrounding environment by combining measurements from various sensors rather than relying on individual sensors, thereby improving driving convenience and safety. With the addition of information and communication technologies (ICT) and cloud technologies, the sensors and related AI algorithms required for autonomous driving are more complex than ever before, not only for individual vehicles but also by remotely accumulating data from numerous vehicle units (fleets) and training AI servers and databases to improve the reliability of vehicle sensor judgments.

[0005] However, in autonomous driving technology that uses radar, the shape of objects is unstable due to the reflective properties of radar sensors. Summary of the Invention

[0006] According to one aspect of this disclosure, a system uses artificial intelligence (AI)-based processing and at least one radar sensor to reliably identify and estimate the positions of objects in a surrounding environment from multiple point data input from the radar sensor. The operation may include: generating at least one cluster having position information and a velocity vector; generating cluster information corresponding to the at least one cluster by clustering the multiple point data input from the radar sensor; correcting the lateral position information by applying a moving average to the lateral position information in the position information; and updating trajectory information including heading information by recalculating the lateral velocity of the velocity vector based on the position information including the corrected lateral position information.

[0007] According to one embodiment, the operation further includes: after updating all trajectory information, generating updated position information and updated velocity vector by applying a two-dimensional Kalman filter to the position information and velocity vector.

[0008] According to one embodiment, the longitudinal velocity of the velocity vector can be used without correcting the measurements of the radar sensor.

[0009] According to one embodiment, the longitudinal position information in the position information can be updated based on the longitudinal velocity obtained by the radar sensor.

[0010] According to one embodiment, when the number of measurements accumulated by the radar sensor is equal to or greater than a threshold, the moving average may include multiple moving averages obtained based on the radar sensor measurement results, wherein the multiple moving averages are generated by applying multiple moving averages with different input sizes.

[0011] According to one implementation, multiple weighted moving averages can be obtained by applying different weights to each of the multiple moving averages based on the rate of change of the current trajectory.

[0012] According to one implementation, when the rate of change of the current trajectory is less than or equal to a specified baseline value, a larger weight is assigned to the moving average value generated by the moving average value with a larger input size, and a smaller weight is assigned to the moving average value generated by the moving average value with a smaller input size.

[0013] According to one implementation, when the rate of change of the current trajectory exceeds a specified baseline value, a smaller weight is assigned to the moving average value generated by the moving average value with a larger input size, and a larger weight is assigned to the moving average value generated by the moving average value with a smaller input size.

[0014] According to one embodiment, the rate of change of the current trajectory may include a value representing the speed at which the target object tracked by the radar sensor moves in the lateral direction.

[0015] According to one implementation, when the number of multiple moving averages is less than a specified number, the object can be considered not to be tracked by the radar sensor.

[0016] According to another aspect of this disclosure, a method for identifying and reliably estimating the position of an object in a surrounding environment based on detection point data obtained from at least one autonomous driving sensor including at least one radar sensor includes: generating at least one cluster having position information and velocity vector; generating clustering information corresponding to the at least one cluster by clustering multiple point data input from the radar sensor; correcting the lateral position information by applying a moving average to the lateral position information in the position information; and updating trajectory information including heading information by recalculating the lateral velocity of the velocity vector based on the position information including the corrected lateral position information. Attached Figure Description

[0017] The above and other objects, features and advantages of this disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0018] Figure 1 This is a block diagram illustrating an example of an overall system for automatically identifying objects and controlling vehicles to achieve purposes such as autonomous driving, according to an embodiment of the present disclosure.

[0019] Figure 2 This is a flowchart illustrating an example of an entire algorithm related to radar signal processing (RSP) according to an embodiment of this disclosure;

[0020] Figure 3 This is a flowchart illustrating an example of an algorithm according to embodiments of the present disclosure that enables the estimation and tracking of the stable position of an object with an unstable shape as observed by a radar sensor.

[0021] Figure 4 This is a diagram illustrating an example of the relationship between the generated clustering location information and the ground truth (GT);

[0022] Figure 5 This is a figure showing example experimental results, including longitudinal and lateral velocity estimation results from radar sensors;

[0023] Figure 6 This is a diagram illustrating a first example of a process for correcting lateral position errors based on a moving average according to an embodiment of the present disclosure;

[0024] Figure 7 This is a diagram illustrating a second example of a process for correcting lateral position errors based on a moving average according to an embodiment of the present disclosure;

[0025] Figure 8 This is a flowchart illustrating an example of measuring the rate of change of a trajectory during the process of correcting lateral position errors based on a moving average according to an embodiment of the present disclosure.

[0026] Figure 9 This is a diagram illustrating an example of the physical meaning of the difference in the magnitude of the moving average input during the process of correcting lateral position errors based on moving averages according to an embodiment of the present disclosure.

[0027] Figure 10 This is a graph illustrating example experimental results of a process for correcting lateral position errors based on moving averages according to embodiments of the present disclosure;

[0028] Figure 11 This is a block diagram illustrating an example of a computing system for autonomous vehicle control and object recognition operations according to an embodiment of the present disclosure. Detailed Implementation

[0029] Embodiments of this disclosure provide a system and method for tracking objects using artificial intelligence (AI) and autonomous driving sensors. In some embodiments, an object tracking system and method are provided that can estimate the position of an object based on measurements of unstable shapes detected by autonomous driving sensors (e.g., radar sensors).

[0030] In some implementations, the technique can reliably estimate position and correct velocity even if the shape of the object being observed by radar is unstable.

[0031] In contrast, LiDAR-based techniques can typically infer location by selecting representative points from a bounding box because the shape accuracy is relatively high. However, for other sensors such as radar, even with representative points selected, the shape of the object observed by radar is highly unstable and can result in significant positional errors. Therefore, processing logic needs to be developed to correct these errors. In some scenarios, the processing logic that estimates the object's representative location based on the bounding box shape assumes that the box represents the object's overall shape. However, because the shape of objects observed by radar is often highly unstable, even with representative points selected, the positional error can still be substantial.

[0032] The embodiments of this disclosure can provide a technique for estimating the precise position of an object even when the shape information of the observed object changes significantly, and further propose a technique for estimating precise lateral velocity and heading by recalculating the unstable lateral velocity detection performance of the radar sensor based on the object's position information.

[0033] Therefore, according to embodiments of this disclosure, a stable position can be estimated based on observations of highly unstable shapes, and stable position information of the object can be generated using information including the object's velocity vector and shape (e.g., information received using radar observation data as input). Furthermore, in scenarios where radar observation data results in highly unstable object shapes but a high confidence level in the longitudinal velocity, radar observations can be used to update the longitudinal velocity and position estimates, while the lateral velocity and lateral position are calculated using techniques according to embodiments of this disclosure.

[0034] Furthermore, according to embodiments of this disclosure, the object position can be further estimated by processing logic operations (e.g., using three-step processing logic) to eliminate lateral position errors, and a novel lateral position correction technique using a moving average is described herein to achieve stable position estimation.

[0035] Furthermore, through the detailed description and accompanying drawings of this disclosure, those skilled in the art will be able to understand the various effects of this disclosure in addition to those described above.

[0036] Some embodiments of this disclosure will now be described in detail with reference to the exemplary accompanying drawings. When adding reference numerals to the components of each drawing, it should be noted that even components shown as the same or equivalent in other drawings are denoted by the same numerals. Furthermore, in describing embodiments of this disclosure, detailed descriptions of related known configurations or functions will be omitted if it is determined that such detailed descriptions would interfere with the understanding of the embodiments of this disclosure.

[0037] Furthermore, in describing the components of this disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are used only to distinguish elements, and the nature, order, sequence, and number of elements are not limited by these terms. Moreover, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in common dictionaries should be understood to be consistent with their meaning in the relevant technical context and should not be construed as having an idealized or overly formal meaning unless expressly defined otherwise in the specification of this disclosure. For example, in this disclosure, "object" and "object" have practically the same meaning, and the expressions "object" and "object" can be used interchangeably in this disclosure.

[0038] Figure 1 This is a block diagram illustrating an example of an overall system for automatically identifying objects and controlling a vehicle to achieve purposes such as autonomous driving, according to an embodiment of the present disclosure.

[0039] Reference Figure 1The vehicle control device 100 according to embodiments of the present disclosure can be implemented inside or outside a vehicle, and some components included in the vehicle control device 100 can be implemented inside or outside the vehicle. For example, the vehicle control device 100 can be integrated with the vehicle's internal control unit, or it can be implemented as a separate device and connected to the vehicle's control unit via a separate connection technology. For example, the vehicle control device 100 may also include... Figure 1 Components not shown in the diagram.

[0040] A vehicle control device 100 according to one embodiment may include a processor 110, a sensor 120 (e.g., a LiDAR sensor), and a memory 130. The processor 110, sensor 120, or memory 130 may be electronically and / or operatively coupled to each other via electronic components such as a communication bus.

[0041] In the following text, hardware operably coupled can mean that hardware establishes a direct or indirect connection with each other in a wired or wireless manner, such that the first piece of hardware can control the second piece of hardware.

[0042] Despite Figure 1 Different boxes are shown in the figures, but the implementation is not limited to these. Figure 1 Some of the hardware may be included in a single integrated circuit (e.g., a system-on-a-chip (SoC)). The type and / or quantity of hardware included in the vehicle control unit 100 are not limited to... Figure 1 The type and / or quantity shown. For example, vehicle control unit 100 may include only... Figure 1 The hardware shown is a portion of the hardware.

[0043] A vehicle control device 100 according to one embodiment may include hardware for processing data based on one or more instructions. For example, the hardware for processing data may include a processor 110. For example, the hardware for processing data may include an arithmetic logic unit (ALU), a floating-point unit (FPU), a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), and / or an application processor (AP). The processor 110 may have a single-core processor architecture or a multi-core processor architecture including dual-core, quad-core, hexa-core, or octa-core processors.

[0044] The processor 110 according to one embodiment may include at least one of a graphics processing unit (GPU), a neural processing unit (NPU), or any combination thereof. For example, a GPU may be referred to as a visual processing unit (VPU). For example, an NPU may be referred to as a neural network processing unit.

[0045] A vehicle control device 100 according to one embodiment may include a sensor 120 for detecting external objects, such as a depth sensor. For example, the depth sensor for detecting external objects may include at least one of a time-of-flight (ToF) sensor, a LiDAR sensor, a structured light sensor, an ultrasonic sensor, an infrared sensor, a radio detection and ranging (radar), an optical distance sensor, or any combination thereof. For ease of explanation, LiDAR will be emphasized below, but the embodiments are not limited thereto.

[0046] According to one embodiment, a vehicle control device 100 may include a LiDAR sensor 120 (or simply LiDAR 120) that acquires multiple points based on pulsed laser signals. For example, LiDAR 120 may acquire a dataset identifying objects surrounding the vehicle control device 100 (or a vehicle including the vehicle control device 100). For example, LiDAR 120 may identify at least one of the following: the position, direction of motion, speed, or any combination thereof of surrounding objects, based on pulsed laser signals emitted from LiDAR 120 reflected back from surrounding objects.

[0047] For example, LiDAR 120 can obtain a dataset representing external objects in a space formed by the x-axis, y-axis, and z-axis based on pulsed laser signals reflected from surrounding objects. For example, LiDAR 120 can obtain a dataset containing multiple points in a space formed by the x-axis, y-axis, and z-axis based on receiving pulsed laser signals at specified time intervals. For example, the multiple points can include points representing external objects within a 3D virtual coordinate system. The 3D virtual coordinate system can include at least one of a vehicle coordinate system, a LiDAR coordinate system, or any combination thereof. However, the 3D virtual coordinate system is not limited to the examples described above.

[0048] The memory 130 of a vehicle control device 100 according to one embodiment may include hardware components for storing data and / or instructions input to and / or output to the processor 110. For example, the memory 130 may include volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM)).

[0049] For example, volatile memory may include at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, pseudo SRAM (PSRAM), or any combination thereof. For example, non-volatile memory may include at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, optical disk, solid-state drive (SSD), embedded multimedia card (eMMC), or any combination thereof.

[0050] The memory 130 of the vehicle control unit 100 may store one or more instructions (or commands) that instruct the processor 110 of the vehicle control unit 100 to perform calculations and / or operations on data. A collection of one or more instructions may be referred to as a program, firmware, operating system, process, routine, subroutine, and / or application.

[0051] In the following text, "application installed in vehicle control unit 100" may mean that one or more instructions provided in the form of an application are stored in memory 130, and that one or more applications are stored in a format executable by the processor 110 of vehicle control unit 100 (e.g., a file with an extension specified by the operating system of vehicle control unit 100).

[0052] For example, memory 130 may include a first neural network model for detecting objects. For example, memory 130 may include a second neural network model for outputting the type and / or score of multiple points obtained by LiDAR 120.

[0053] In one embodiment, the processor 110 may obtain at least one of a first virtual bounding box representing a target object, a first category representing the type of the target object, or a combination thereof, based on multiple points obtained by LiDAR 120 and a first neural network model stored in memory 130.

[0054] For example, processor 110 can obtain at least one of the following: a first virtual bounding box representing a target object, a first category representing the type of the target object, or any combination thereof, based on inputting multiple points into a first neural network model. For example, the first neural network model may include an object detection model. For example, the target object may include an external object located within a specified distance from vehicle control device 100 (or the vehicle including vehicle control device 100). For example, the target object may include an object identified and tracked (e.g., continuously tracked) by vehicle control device 100. For example, the type of the target object may be one or more of a variety of types used for classifying the target object. For example, the type of the target object may include at least one of a first type representing the ground, a second type representing a type different from the ground, or any combination thereof. However, the type of the target object is not limited to the types described above. For example, the type of the target object may include at least one of a third type representing a person, a fourth type representing a vehicle, or any combination thereof, but is not limited thereto.

[0055] In one embodiment, the processor 110 may obtain, based on a plurality of points and a second neural network model, at least one of the following: a first part of the points corresponding to at least a portion of the target object, a second category indicating the type of the target object identified by the first part of the points, or any combination thereof. For example, the second neural network model may include a segmentation model.

[0056] For example, the second neural network model may include a neural network model for obtaining the type of multiple points and the score of multiple points.

[0057] For example, processor 110 can obtain a first set of points that correspond to at least a portion of the target object by inputting multiple points into a second neural network model. For example, processor 110 can identify the types of multiple points by inputting multiple points into the second neural network model. For example, processor 110 can obtain a first set of points that correspond to at least a portion of the target object based on the type of each point in the multiple points.

[0058] In one embodiment, processor 110 may execute a first specified algorithm on a plurality of points. For example, processor 110 may execute a first specified algorithm on the plurality of points to classify the type of each point among the plurality of points. For example, processor 110 may classify a second subset of points among the plurality of points that correspond to a specified type. For example, the specified type may include a type representing the ground.

[0059] For example, processor 110 may classify a second part of points corresponding to a specified type based on performing a first specified algorithm on multiple points, and obtain (or identify) the first part of points by excluding the second part of points from the multiple points.

[0060] In one embodiment, processor 110 may obtain at least one of the following: a partial category for obtaining a second category, a score for each of the multiple points, or any combination thereof, based on inputting multiple points into a second neural network model. For example, processor 110 may obtain a partial category and a score for each of the multiple points based on inputting multiple points into the second neural network model. For example, the partial category may include classifying each of the multiple points into an arbitrary type.

[0061] For example, processor 110 may fuse partial categories, scores for each point among multiple points, and a second set of points. For example, processor 110 may perform clustering based on the fusion of partial categories, scores for each point among multiple points, and the second set of points. For example, clustering may include grouping first set of points corresponding to at least a portion of a target object.

[0062] For example, processor 110 can obtain a point cloud for generating a second virtual bounding box based on the first subset of points. For example, processor 110 can obtain the point cloud based on grouping the first subset of points.

[0063] For example, processor 110 can generate a second virtual bounding box for representing a target object based on a point cloud. This second virtual bounding box is different from the first virtual bounding box. For example, the second virtual bounding box may include a box containing at least some of the first portion of points.

[0064] For example, the processor 110 can identify a heading that indicates the direction of travel of a target object based on at least one of a first portion of points, a point cloud, or any combination thereof.

[0065] For example, processor 110 can identify the position of the second virtual bounding box in the virtual coordinate system based on at least one of the first partial points, point cloud, or any combination thereof. For example, processor 110 can identify the size of the second virtual bounding box based on at least one of the first partial points, point cloud, or any combination thereof. For example, processor 110 can identify a second category based on at least one of the first partial points, point cloud, or any combination thereof. For example, processor 110 can identify at least one of the following: the heading indicating the direction of movement of the target object, the position of the second virtual bounding box in the virtual coordinate system, the size of the second virtual bounding box, the second category, or any combination thereof, based on at least one of the first partial points, point cloud, or any combination thereof. For example, processor 110 can identify the heading of the bounding box based on at least one of the first virtual bounding box, the first category, the heading of the second virtual bounding box, the position of the second virtual bounding box, the size of the second virtual bounding box, the second category, or any combination thereof. For example, processor 110 can identify the position of the bounding box in the virtual coordinate system based on at least one of the first virtual bounding box, the first category, the heading of the second virtual bounding box, the position of the second virtual bounding box, the size of the second virtual bounding box, the second category, or any combination thereof. For example, processor 110 can obtain a third category indicating the type of the target object corresponding to the bounding box based on at least one of the first virtual bounding box, a first category, the heading of the second virtual bounding box, the position of the second virtual bounding box, the size of the second virtual bounding box, the second category, or any combination thereof. For example, processor 110 can obtain the heading of the bounding box, the position of the bounding box in the virtual coordinate system, a third category indicating the type of the target object corresponding to the bounding box, or at least one combination thereof, based on at least one of the first virtual bounding box, a first category, the heading of the second virtual bounding box, the position of the second virtual bounding box, the size of the second virtual bounding box, the second category, or any combination thereof.

[0066] For example, processor 110 may assign a first identifier to the second virtual box for tracking the second virtual box. For example, processor 110 may assign a second identifier corresponding to the first identifier to the bounding box.

[0067] For example, processor 110 can use the second identifier to track bounding boxes. For example, processor 110 can track a target object based on identifying multiple bounding boxes, including those assigned the second identifier, across multiple frames. For example, in a scenario where the second identifier is an identifier assigned to the bounding box corresponding to the target object, processor 110 can track the target object by identifying multiple bounding boxes assigned the second identifier across multiple frames.

[0068] In one embodiment, the processor 110 may output a bounding box corresponding to the target object based on at least one of a first virtual box, a first category, a first partial point, a second category, or any combination thereof. For example, the bounding box may include an example of the target object represented in hexahedral form in a virtual coordinate system.

[0069] The operations performed by the CPU, GPU, and / or NPU included in processor 110 will be briefly described below.

[0070] In one embodiment, the processor 110 may include at least one of a CPU, GPU, NPU, or any combination thereof. For example, at least one of a GPU, NPU, or any combination thereof may obtain a first virtual bounding box and a first category based on a first neural network model. For example, at least one of a GPU and NPU may obtain a first virtual bounding box and a first category. For example, at least one of a GPU, NPU, or any combination thereof may obtain a partial category for obtaining a second category and a score for each point among a plurality of points based on a second neural network model. For example, at least one of a GPU and NPU may obtain a partial category for obtaining a second category and a score for each point among a plurality of points based on a second neural network model. For example, the CPU may classify a second subset of points corresponding to a specified type among the plurality of points by performing a first specified algorithm on the plurality of points to classify the type of each point among the plurality of points.

[0071] As described above, a vehicle control device 100 according to one embodiment may include at least one processor 110. The vehicle control device 100 can accurately detect a target object by using at least one processor 110 to detect the target object. Furthermore, by performing parallel processing, the vehicle control device 100 can reduce the load on each processor.

[0072] The following sections will describe object recognition, including examples based on specific scenarios. Specifically, to illustrate the general concepts of AI-based object recognition, a LiDAR-based object recognition scenario will be described.

[0073] In some implementations, the object recognition process of the LiDAR 120 and AI module includes three operations: preprocessing, segmentation, and tracking.

[0074] Before performing object recognition functions, the object recognition system 1000 (for example, see...) Figure 7 Preprocessing can be performed. Preprocessing may include, for example, removing points that constitute the ground based on laser sensing data input from the LiDAR 120 (i.e., the unprocessed raw data). Since laser beams reflected from the ground may be misidentified as objects on the ground, the process of distinguishing between ground and non-ground elements can be performed in the preprocessing operation, and in some cases, it can also be performed in the segmentation operation.

[0075] In some implementations, preprocessing in AI object recognition may include a process in which the image processing tool of the AI ​​module in processor 110 removes noise from the LiDAR point cloud image and reduces the total number of points present in the LiDAR point cloud image by, for example, voxel downsampling techniques to improve computational efficiency.

[0076] For reference, LiDAR point cloud images can also be displayed as bird's-eye view (BEV). When a LiDAR image is created to resemble a city viewed from above, it is called a BEV image.

[0077] For example, as described above, LiDAR 120 emits laser beams into the surrounding environment and records the round-trip time of the laser beams being reflected by external objects and returning, thereby generating a small dot (i.e., a point) for each of a large number of laser signals, based on which the distance to that point can be calculated. By repeatedly emitting a large number of laser beams in this manner, processor 110 can generate a real-time LiDAR map of the surrounding environment as a three-dimensional map of the BEV type, and in some cases, a two-dimensional map.

[0078] The black-and-white lines or surfaces on a LiDAR point cloud image can be composed of a large number of points (e.g., each point is generated by a laser beam from a LiDAR120), for this reason, LiDAR-sensed images are also called LiDAR point cloud images. In some implementations, LiDAR point cloud images can be reconstructed in color by combining a red-green-blue depth (RGB-D) sensor with a LiDAR sensor.

[0079] While it's difficult for humans to identify objects from a single point in a LiDAR point cloud image, when human observers view the point cloud from a BEV perspective or a two-dimensional planar view, they can gain a general understanding of what the surrounding environment of an autonomous vehicle looks like. Furthermore, they can identify objects such as vehicles, buses, pedestrians, street trees, and traffic signs within the LiDAR point cloud image. In AI image recognition technology, things or people can be called objects, and each object can be categorized into groups belonging to specific characteristics, such as vehicle categories, bus categories, etc.

[0080] In some implementations, machine learning (e.g., machine learning involving deep AI neural networks) can be implemented to classify objects in LiDAR point cloud images into categories such as cars, buses, etc. In some cases, AI training can be performed before deploying the AI ​​neural network (e.g., as a prior step) to locate objects in the LiDAR point cloud image and use the AI ​​neural network to identify the categories of those objects.

[0081] For example, a name PANDASET TM The dataset (source: https: / / pandaset.org / #data-collection) contains over 48,000 camera images (mostly taken in Silicon Valley, USA) and over 16,000 LiDAR scan images, labeled in a total of 28 categories, including pedestrians, vehicles, bicycles, construction site signs, traffic signs, etc.

[0082] In addition, point cloud processing tools (such as Open3D) can be used. TM (Source: https: / / www.open3d.org / ) This allows for the visualization of LiDAR point cloud images to meet user expectations. Because the LiDAR 120 is capable of distance detection, 3D LiDAR images can be reproduced more realistically, for example, by using Open3D... TM During visual processing, distant objects are displayed in dark blue, while nearby objects are displayed in light blue.

[0083] Furthermore, as mentioned above, in some implementations, the raw LiDAR image (i.e., the raw data) can be preprocessed, for example, by applying a technique called voxel (3D pixel) downsampling to the LiDAR point cloud image. In this case, a voxel refers to a cubic 3D pixel, and voxel downsampling is a technique that reduces the number of points, thus requiring less AI computation, while preserving the structure of the various objects contained in the LiDAR point cloud.

[0084] In some implementations, the LiDAR 120 emits, for example, "m" laser beams "n" times during a scan cycle. In this case, the scan values ​​of the laser beams reflected back from external objects can be represented as an (m x n) matrix, and the (m x n) matrix data is called a range image. Each point constituting the LiDAR point cloud image can contain, for example, depth (i.e., distance) information, as well as the intensity, azimuth, tilt angle, and other additional information of the reflected laser pulses. As an example, for the range image, a device such as Waymo can be used. TMAI learns from large datasets such as Open Datasets (WOD).

[0085] Distance View (RV) is a technique that transforms a 3D point cloud into a 2D or 2.5D scene and represents it as a LiDAR 3D image that is intuitively understood by humans, much like a simulated picture rather than a large number of points. In a distance view image, the 3D LiDAR point cloud image has two-dimensional coordinates, but 3D laser-related information (angle, tilt, intensity, etc.) recorded when acquiring the distance image is not discarded. A 2D distance view image is created by applying a variable called width to the (x, y) coordinates of the 3D LiDAR image, obtaining one 2D axis coordinate, and by applying a variable called height and distance image information representing distance (depth) to the (z) coordinate, obtaining another 2D axis coordinate.

[0086] Furthermore, the AI ​​algorithm according to embodiments of this disclosure may include a convolutional neural network (CNN), which can be trained, for example, by an AI training module, to extract features (or feature points) from image data. For example, training may utilize a dataset containing tens of thousands of images, and CNNs are capable of processing images ranging from one-dimensional to three-dimensional. Therefore, the results of a distance-view image processing tool can be used to train a CNN neural network and perform functions that help AI accurately identify objects in images.

[0087] In scenarios where machine learning is used to identify objects around autonomous vehicles, the aforementioned operation of generating ground truth (GT) bounding boxes on LiDAR images can be implemented as part of object recognition. In machine learning, ground truth (GT) is a term used to indicate the original or actual value of the data that the AI ​​is learning. For example, GT can be considered an image annotation applied to LiDAR point cloud images, typically a bounding box with a box-shaped boundary.

[0088] For example, to perform object recognition, the AI ​​module can determine labels and group various objects. In some cases, the spacing of the 3D data points used to output the ground truth (GT) bounding box can be configured so that, for example, a GT bounding box contains approximately 50 to 1000 LiDAR cloud points.

[0089] In some cases, such GT annotations are not present in the raw data captured by sensors such as LiDAR 120 while the vehicle is in motion. Processor 110 can identify objects belonging to various categories (e.g., road signs, crosswalks, pedestrians, other vehicles, and center lanes) as objects. GT annotations can be used to measure object recognition error, and in some implementations, AI performance is evaluated by comparing the object identification results of the AI ​​algorithm of processor 110 with the real-world situation. GT bounding boxes can be applied to the raw image as annotations, which can be set manually by the user or through GT calculation tools (e.g., grid-striding).

[0090] AI object recognition can also utilize predicted bounding boxes. A predicted bounding box can represent the result of processor 110 identifying objects of a specific category from raw image data obtained from LiDAR 120, etc. The predicted bounding box can be represented as another bounding box similar to the ground truth (GT) bounding box. However, unlike the GT bounding box, the predicted bounding box is the result of AI computation for autonomous driving. The predicted bounding box can coincide with the GT bounding box, but it can also not coincide with the GT bounding box, or it can be completely non-overlapping with the GT bounding box.

[0091] For reference, since it is difficult to determine precisely whether an object of a particular category actually exists at a particular location based solely on the predicted bounding boxes, these predicted bounding boxes are often referred to as probability boxes (P-Boxes).

[0092] Segmentation can be performed after preprocessing. Segmentation can, for example, cause specific portions of a road (e.g., traffic lights) to be displayed with a specific annotation (e.g., red), while other portions (asphalt road) are displayed with another annotation (e.g., blue). Segmentation can involve clustering the point cloud into specified groups and generating P-Boxes. In some implementations, cluster expansion can be performed, where the expansion target includes all points within a given distance from the seed point plus an additional incremental distance ε (where the given distance constitutes the unexpanded cluster).

[0093] In some implementations, point cloud-based clustering and P-Box generation can be performed during the segmentation process. For example, point labels can be obtained from data collected by a sensor (e.g., LiDAR 120) using an AI network that performs the segmentation.

[0094] The embodiments of this disclosure can utilize rule-based ground recognition and label fusion technology, which helps to solve the problem of ground recognition errors that occur during segmentation. This road surface recognition algorithm can apply various techniques, including slope-based road surface recognition, grid-based road surface recognition, and other non-planar road surface recognition. The embodiments of this disclosure are described according to slope-based road surface recognition technology when recognizing road surfaces within tunnels.

[0095] For reference, semantic segmentation can involve assigning a unique category label to each point in the point cloud generated by LiDAR 120. In LiDAR image processing technology, semantic segmentation can find and use meaningful information from LiDAR data for object recognition and scene reconstruction, thereby enabling autonomous driving. Various semantic segmentation AI models can be used, such as projection-based methods, point-based methods, sparse convolution-based methods, etc. For example, the semantic segmentation result can be the result of AI operations performed by processor 110 using the NVIDIA DRIVE™ AGX system, and with such a configuration, for example, various colors can be added to the LiDAR point cloud image.

[0096] After performing the segmentation process described above, in some implementations, the LiDAR image can be further processed through post-processing. Post-processing may involve converting the point cloud data into a 3D map or model, which can be meaningful information for autonomous driving. In some cases, post-processing may also include removing noise and errors from the LiDAR point cloud image, identifying objects such as vehicles or pedestrians from the point cloud, and in some cases, assigning a unique identifier to the point cloud information to register it.

[0097] Figure 2 This is a flowchart illustrating an example of an entire algorithm related to radar signal processing (RSP) according to an embodiment of this disclosure.

[0098] Reference Figure 2 In operation S100, the detected data is input from the radar sensor and a preprocessing process is performed according to the processor 110 (e.g., including an AI module) and computing system 1000 of this disclosure embodiment.

[0099] In some cases, the preprocessing procedure of operation S100 is the same as the preprocessing procedure described above. For example, operation S100 may include processing detection point data received from each of a plurality of radars (e.g., five radars) appropriately placed around the vehicle. In some embodiments, if the data is located in different radar sensor coordinate systems, the data can be processed to transform it into a single data format in the vehicle's own coordinate system equipped with the radar sensors. In some cases, the radar detection points may contain a significant amount of noise. Therefore, in some embodiments, noise filtering may also be performed in operation S100, for example, using signal information provided by the radar sensors and data information from the vicinity of the detection points.

[0100] exist Figure 2 In operation S200, a clustering process is performed. Clustering can provide many technical advantages; for example, by clustering detection points for which noise has been removed in operation S100, the AI ​​computation time for object tracking using radar can be reduced.

[0101] In some implementations, point cloud clustering can be performed using both velocity information (e.g., using radio waves and the Doppler principle) and location information, leveraging the characteristics of sensors such as radar. In some cases, unclustered points can be filtered out as invalid detection points.

[0102] In operation S300, prediction can be performed. For example, the following process can be performed: an object detection (OD) network (e.g., included in an AI module) according to an embodiment of this disclosure predicts objects based on the clustering results generated in operation S200. For example, the aforementioned P-Box can be generated in operation S300. In this case, the P-Box information generated in operation S300 can be utilized in subsequent operation S400.

[0103] In operation S400, clustering information is generated. This clustering information may include, for example, longitudinal velocity, lateral velocity, heading, position, and cluster box information. In some cases, due to the characteristics of the radar sensor, the detection points contained in each cluster have Doppler information, which can be analyzed in operation S400 to generate representative velocity vectors. Furthermore, in operation S400, the position information of the detection points contained in the clusters can be used to generate representative positions and bounding boxes for the clusters.

[0104] In operation S500, the association between clusters and trajectories is performed. For example, an association operation can be performed between (i) the trajectory predicted at the current time point based on data updated to the previous time point and (ii) the clusters newly observed at the current time point. The trajectory can be predicted by using the position and velocity information of the previous time point to predict the position at the current time point (e.g., through a trajectory preprocessing procedure).

[0105] In operation S600, trajectory prediction is performed. In this case, the predicted trajectory information may include trajectory position, trajectory box shape, trajectory velocity, trajectory heading, duration (age), confidence level, etc. The trajectory prediction X(t | t-1) is a prediction of the trajectory at the current time t based on information collected up to the previous time t-1. For example, a state transition model can be used to generate the prediction X(t | t-1) from X(t-1) using previously collected clustering information (e.g., velocity, heading, and position information) up to time t-1.

[0106] In operation S700, a tracking result is generated. For example, for C(t) already associated with the predicted trajectory X(t | t-1), the trajectory information can be updated using the new clustering information at the current time. For example, this process can use Kalman filtering to update the trajectory to the current time t using information collected up to the current time t. For example, the updated trajectory information may include trajectory position, trajectory box shape, trajectory velocity, trajectory heading, duration, confidence level, etc., as described above. Further details of the tracking operation in S700 will be described below with reference to various examples of embodiments of this disclosure.

[0107] Figure 3 This is a flowchart illustrating an example of an algorithm 300, according to embodiments of the present disclosure, that enables the estimation and tracking of the stable position of an object with an unstable shape observed by a radar sensor. For example, Figure 3 The algorithm 300 shown can be used to obtain stable position information from radar observations with highly unstable shapes. In some implementations, algorithm 300 can be implemented to perform... Figure 2 Tracking operations in S700.

[0108] In description Figure 3 Previously, it's important to note that the instability of observation results can stem from a variety of reasons. For instance, in radar observation scenarios, unlike LiDAR sensors, there may be issues where radar data cannot be accurately reflected from the object's outline. For example, radar signals might be reflected irregularly, in which case the clustering process might divide the data into clusters, resulting in only a very small number of radar measurement points. In such cases, even if clustering and shape generation are performed, the probability of lateral and longitudinal positional errors is extremely high.

[0109] In the following text Figure 3 The description will be combined intermittently as appropriate. Figures 4 to 9 right Figure 3 The different operations in the process are described to describe Figure 3 Further details of each operation.

[0110] exist Figure 3 In this implementation, clustering information can be one of the inputs and can include position and velocity vectors. In some radar observation scenarios, the longitudinal component of the velocity vector (x, y) may be measured very accurately, but the lateral component of the velocity may be very inaccurate. The position (x, y) coordinates may have large position errors in both the lateral and longitudinal directions, depending on the size of the cluster distribution. To mitigate such problems, the position information can be corrected, and lateral velocity correction can be performed using the lateral position displacement difference.

[0111] Figure 4 This is a graph illustrating an example of the relationship between the generated clustering location information and the ground truth (GT). The clustering location information (e.g., measurements) may deviate to some extent from the target location GT and the curve connecting it. For example, Figure 5 This is a figure showing the experimental results 400, including the longitudinal velocity estimation results 410 and the lateral velocity estimation results 420 from the radar sensor.

[0112] Refer again Figure 3 In operation S10, the clustering information and previous tracking results described above can be obtained (e.g., input into the AI ​​module). In operation S10, a weighted multiple moving average can be calculated.

[0113] In some implementations, a moving average (MA) can be used for position correction. For example, in the case of longitudinal position, where the longitudinal velocity value observed by radar is accurate, the observation results can be used as is to update the longitudinal position. In this case, in operation S10, only the lateral position is used to calculate the correction value of the lateral position using the moving average technique. In some implementations, the input size of the moving average can be adaptively changed, for example, it can be determined according to the trajectory duration (e.g., if the trajectory duration is at least 10, the size of the moving average window can be as high as 10).

[0114] Figure 6 This is a diagram illustrating a first example of a process for correcting lateral position errors based on a moving average according to an embodiment of the present disclosure. Figure 6 As shown in the implementation, the radar observation position P at the current time point is calculated based on the average obtained by summing the observation results to the previous time point t-4. t Correct to P' t That is, in Figure 6 In the example, the moving average window includes time points t, (t-1), (t-2), (t-3), and (t-4).

[0115] Figure 7This is a diagram illustrating a second example of a process 500 for correcting lateral position errors based on a moving average according to an embodiment of the present disclosure. In this example, Figure 7 It shows in Figure 3 The process of applying weighted multiple moving average in operation S10.

[0116] like Figure 7 As shown, for a trajectory that has accumulated sufficient duration, moving averages (MA) of various sizes can be obtained. Figure 7 In the example, the box in the upper left corner shows a single MA value obtained using 5 inputs. The box to its right shows multiple MA values ​​obtained. In this example, four different MA values ​​were obtained using 6 inputs, 4 inputs, 2 inputs, and 1 input (each input is indicated by different brackets in the box). Based on these different moving averages (each value can be obtained using a different number of input points to obtain the MA), the final lateral position correction can be performed by taking a weighted average of the different MA values ​​(e.g., applying different weights to different MA values). In some implementations, the update rate can be adjusted, for example by adjusting the weights of the different moving averages according to the rate of change of the current trajectory. More details on adjusting the update rate will refer to [reference needed]. Figure 9 To explain.

[0117] exist Figure 7 In the diagram, reference numeral 510 indicates a cluster location, such as a radar measurement result, and reference numeral 511 indicates the individual location information based on the radar measurement result 510. Reference numeral 520 represents the most recent estimated target location, which can be used to generate the input to a Kalman filter to generate the estimated target location for the next time point. Reference numeral 521 represents a past target location estimate, which is a correction result of the location information such as 511. The dashed line represents the GT reference point. According to an embodiment of this disclosure, instead of using only the most recent measurement point (in measurement cluster 510) to perform Kalman updates, different MA results are obtained using a different number of past measurement points in 510, a weighted average of these different MA results is generated, and this weighted average is used as a new measurement result. This new measurement result can then be combined with the previously estimated target location for Kalman filter updates. By appropriately adjusting the size of the different MA operations and / or by adjusting the weights assigned to the different MA results, more stable and accurate tracking results can be obtained.

[0118] Figure 8This is a flowchart 600 illustrating an example of measuring the rate of change of a trajectory during a process of correcting lateral position errors based on moving averages according to an embodiment of the present disclosure. The rate of change of the current trajectory can be a numerical value indicating how fast the lateral position of the trajectory changes. The measured rate of lateral change can then be used to determine the appropriate magnitude of different MA operations and / or the appropriate weights assigned to different MA results to produce accurate and stable tracking.

[0119] exist Figure 8 In the example, operation S1 is a process for determining whether a target is detected, for example, based on a point count exceeding 10. If a target is detected, it is determined whether the change in lateral movement is large or small. In some implementations, the difference between MA results for different window sizes can be determined. For example, operation S2 is a process for determining whether the difference between an MA with an input size of 10 and an MA with an input size of 2 exceeds a threshold (i.e., "yes" in operation S2), while operation S3 is a process for comparing an MA with an input size of 10 and an MA with an input size of 1. If the difference between the MA results for different input sizes in S2 and / or S3 is large, in some implementations, the lateral velocity gradient can be further determined and compared with a threshold, as shown in operation S4. If the lateral velocity gradient exceeds the threshold, the target can be classified as a movement-2 case; otherwise, it can be classified as a movement-1 case.

[0120] Based on the results of process 600, the trajectory update rate can be changed; for example, the trajectory can be updated more slowly or more quickly. For instance, the trajectory update rate can be adjusted by changing the weights assigned to different MA results in the weighted average. Figure 8 In a trajectory where lateral position changes slowly—for example, if the trajectory is considered "stable"—the update rate can be reduced, for instance, by assigning larger weights to MA results with larger inputs and smaller weights to MA results with smaller inputs. Conversely, if lateral position changes rapidly, such as in "Move-1" or "Move-2" trajectories, the update rate can be increased, for instance, by assigning smaller weights to MA results with larger inputs and larger weights to MA results with smaller inputs. Furthermore, comparisons... Figure 8 In the context of the Movement-1 and Movement-2 scenarios, the lateral rate of change of the trajectory in the Movement-2 scenario is greater than that in the Movement-1 scenario. In this case, the weighting between the Movement-1 and Movement-2 scenarios can be further differentiated, for example, by proportionally assigning a larger weight to the smaller input MA result in the Movement-2 scenario to better capture rapidly changing local movements.

[0121] Figure 9This is a schematic diagram of example 700 illustrating the physical meaning of the difference in the input magnitude of the moving average during the process of correcting lateral position errors based on a moving average according to an embodiment of the present disclosure.

[0122] like Figure 9 As shown, (a) illustrates point information measured by a sensor (e.g., a radar sensor) over time; (b) shows different MA plots under different MA input sizes (i.e., different numbers of points input for different MA results). Subsequently, in (c), a weighted multiple moving average (MA) operation is performed, wherein the different MA results in (b) are weighted (e.g., using different weights), and a weighted multiple moving average is calculated as shown in (d).

[0123] As this example illustrates, as the MA input size increases, the resulting average reflects a more global position update value and is less affected by local disturbances. Conversely, when the MA input size is small, the resulting average reflects a local position update value and is more affected by local disturbances, thus responding quickly to changes.

[0124] Refer again Figure 3 In operation S20, the trajectory position is generated based on a weighted multiple moving average. In operation S30, the lateral velocity is recalculated and generated based on the estimated path. In this case, the lateral velocity can be estimated by calculating the lateral velocity using the object's position displacement.

[0125] In operation S40, the velocity vector of the current object is extracted using longitudinal and lateral velocities, and the heading angle of the trajectory is calculated using the velocity vector to generate correction information.

[0126] In operation S50, when all trajectory information is updated, it is passed through a two-dimensional Kalman filter to perform final corrections on velocity and position, thereby completing the object tracking logic. For example, in the Kalman filter update step of S50, the updated trajectory information can be combined with previous tracking results (e.g., it can be used to obtain a prediction of the current trajectory) to obtain an updated estimate of the tracking result at the current time point.

[0127] Figure 10 This is a figure illustrating example experimental results 800 of a process for correcting lateral position errors based on moving average according to an embodiment of the present disclosure.

[0128] According to embodiments of this disclosure, stable position estimation can be achieved through processing logic (e.g., using the three-step method described above), which stabilizes the position of objects with unstable shape observations. For example, in Figure 10In the case of clustering information shown in (a), the lateral position and velocity are very unstable, but as shown in (b), by utilizing the technology according to embodiments of the present disclosure, interference can be reduced and stable tracking can be achieved.

[0129] like Figure 10 As shown, embodiments of this disclosure can provide a tracking technique that can generate stable tracking results from observations with highly unstable shapes. Unlike LiDAR sensors, which provide point data reflected from the contours of an object, radar sensors may have uncertain reflection locations that may not precisely match the object's contours. Therefore, when extracting the shape of an object using radar data obtained in this way, it is highly likely that the extracted shape will differ from the actual object's shape information, and the extracted shape may also have significant positional errors. According to embodiments of this disclosure, longitudinal velocity and position estimates can be updated using observations based on a tracking technique that processes radar sensor inputs providing velocity and shape information, while lateral velocity and position are calculated using a novel scheme. In this case, the confidence level of the longitudinal velocity can be increased to a certain level to produce better results. For example, according to embodiments of this disclosure, when updating the lateral position, a novel correction technique can be used that uses different moving averages with various input sizes to achieve stable position estimates. Therefore, embodiments of this disclosure can be used for situations where the detected object shape is highly unstable and are applicable to all systems using tracking techniques, not just those using sensors. However, for sensors without velocity observations, additional processing can be used to correct accuracy.

[0130] Figure 11 This is a block diagram illustrating a computing system 1000 for automatic vehicle control and object recognition operations according to an embodiment of the present disclosure.

[0131] Reference Figure 11 The computing system 1000 may include at least one processor 1100, memory 1300, user interface input device 1400, user interface output device 1500, storage device 1600 and network interface 1700, which are connected via bus 1200.

[0132] Processor 1100 may be a central processing unit (CPU) or a semiconductor device for processing instructions stored in memory 1300 and / or storage device 1600. Memory 1300 and storage device 1600 may include various volatile or non-volatile storage media. For example, memory 1300 may include read-only memory (ROM) and random access memory (RAM).

[0133] Therefore, the processes of the methods or algorithms described in relation to embodiments of this disclosure can be directly implemented by hardware, software modules, or combinations thereof executed by processor 1100. Software modules can reside in storage media (i.e., memory 1300 and / or storage device 1600), such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, solid-state drive (SSD), removable disk, or CD-ROM.

[0134] An exemplary storage medium is connected to processor 1100, and processor 1100 can read information from and write information to the storage medium. In another approach, the storage medium can be integrated with processor 1100. The processor and storage medium can reside in an application-specific integrated circuit (ASIC). The ASIC can reside in a user terminal. In yet another approach, the processor and storage medium can reside as separate components in the user terminal.

[0135] This disclosure presents a technique for stably estimating position and correcting velocity even when the shape of an object observed by radar is unstable.

[0136] Although exemplary embodiments of this disclosure have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions may be made without departing from the scope and spirit of this disclosure.

[0137] Therefore, the exemplary embodiments disclosed herein are for illustrative purposes only and are not intended to limit the technical concept of this disclosure. It should be understood that these exemplary embodiments are not intended to limit the scope of the technical concept of this disclosure. The scope of protection of this disclosure should be understood from the appended claims, and all technical concepts within the equivalent scope should be interpreted as falling within the scope of the claims of this disclosure.

Claims

1. A system comprising: At least one autonomous driving sensor, including at least one radar sensor; At least one processor; and At least one memory stores computer program instructions that, when executed by the at least one processor, perform artificial intelligence (AI) based operations to identify and estimate the positions of objects in the surrounding environment based on detection point data obtained from the at least one radar sensor, the operations including: Preprocessing is performed on the detection point data obtained from the at least one radar sensor to generate preprocessed data; Clustering is performed on the preprocessed data to generate at least one point cluster; Clustering information is generated from the at least one point clustering, wherein the clustering information includes location information and velocity vector; Generate predictions of trajectories updated to the previous time step; Perform a correlation between the clustering information and the predicted trajectory updated to the previous time step; and Based on the clustering information and the trajectory prediction updated to the previous time step, an updated estimate of the trajectory at the current time step is generated. The step of generating the updated estimate of the trajectory includes: Updated lateral position information is generated by applying a moving average to the lateral position information within the clustering information; and The trajectory information, which includes the heading information of the trajectory, is updated by recalculating the lateral velocity of the velocity vector based on the position information containing the updated lateral position information.

2. The system according to claim 1, wherein, The step of generating the updated estimate of the trajectory further includes: After updating all trajectory information, updated position information and updated velocity vector are generated by applying a two-dimensional Kalman filter to the position information and the velocity vector.

3. The system according to claim 1, wherein, The step of generating the updated estimate of the trajectory further includes: The longitudinal velocity of the velocity vector is used without updating.

4. The system according to claim 3, wherein, The step of generating the updated estimate of the trajectory further includes: The longitudinal position information in the position information is updated based on the longitudinal velocity obtained by the at least one radar sensor.

5. The system according to claim 1, wherein, The step of applying a moving average to the lateral location information includes: Based on determining that the number of measurements accumulated by the radar sensor is equal to or greater than a threshold, multiple moving averages are generated based on the radar sensor measurement results, wherein the multiple moving averages are generated by applying multiple moving averages with different input sizes.

6. The system according to claim 5, wherein, The step of applying a moving average to the lateral location information includes: Multiple weighted moving averages are generated by applying different weights to each of the multiple moving averages based on the rate of change of the trajectory.

7. The system according to claim 6, wherein, The steps for generating the multiple weighted moving averages include: Based on the determination that the rate of change of the trajectory is equal to or less than the baseline value, a larger weight is assigned to the moving average generated by the moving average with a larger input size, and a smaller weight is assigned to the moving average generated by the moving average with a smaller input size.

8. The system according to claim 6, wherein, The steps for generating the multiple weighted moving averages include: Based on the determination that the rate of change of the trajectory exceeds the benchmark value, a smaller weight is assigned to the moving average generated by the moving average with a larger input size, and a larger weight is assigned to the moving average generated by the moving average with a smaller input size.

9. The system according to claim 6, wherein, The rate of change of the trajectory includes a numerical value representing the speed at which the target object, tracked by the at least one radar sensor, moves in the lateral direction.

10. The system according to claim 6, wherein, Based on the determination that the number of the plurality of moving averages is less than a specified number, it is considered that the object is not being tracked by the at least one radar sensor.

11. An artificial intelligence (AI)-based method for identifying and estimating the position of objects in a surrounding environment based on detection point data obtained from at least one autonomous driving sensor, including at least one radar sensor, the method comprising the steps of: Preprocessing is performed on the detection point data obtained from the at least one radar sensor to generate preprocessed data; Clustering is performed on the preprocessed data to generate at least one point cluster; Clustering information is generated from the at least one point clustering, wherein the clustering information includes location information and velocity vector; Generate predictions of trajectories updated to the previous time step; Perform a correlation between the clustering information and the predicted trajectory updated to the previous time step; and Based on the clustering information and the trajectory prediction updated to the previous time step, an updated estimate of the trajectory at the current time step is generated. The step of generating the updated estimate of the trajectory includes: Updated lateral position information is generated by applying a moving average to the lateral position information within the clustering information; and The trajectory information, which includes the heading information of the trajectory, is updated by recalculating the lateral velocity of the velocity vector based on the position information containing the updated lateral position information.

12. The method according to claim 11, wherein, The step of generating the updated estimate of the trajectory further includes: After updating all trajectory information, updated position information and updated velocity vector are generated by applying a two-dimensional Kalman filter to the position information and the velocity vector.

13. The method according to claim 11, wherein, The step of generating the updated estimate of the trajectory further includes: The longitudinal velocity of the velocity vector is used without updating.

14. The method according to claim 13, wherein, The step of generating the updated estimate of the trajectory further includes: The longitudinal position information in the position information is updated based on the longitudinal velocity obtained by the at least one radar sensor.

15. The method according to claim 11, wherein, The steps of applying a moving average to the lateral position information include: Based on determining that the number of measurements accumulated by the radar sensor is equal to or greater than a threshold, multiple moving averages are generated based on the radar sensor measurement results, wherein the multiple moving averages are generated by applying multiple moving averages with different input sizes.

16. The method according to claim 15, wherein, The steps of applying a moving average to the lateral position information include: Multiple weighted moving averages are generated by applying different weights to each of the multiple moving averages based on the rate of change of the trajectory.

17. The method according to claim 16, wherein, The steps for generating the multiple weighted moving averages include: Based on the determination that the rate of change of the trajectory is equal to or less than the baseline value, a larger weight is assigned to the moving average generated by the moving average with a larger input size, and a smaller weight is assigned to the moving average generated by the moving average with a smaller input size.

18. The method according to claim 16, wherein, The steps for generating the multiple weighted moving averages include: Based on the determination that the rate of change of the trajectory exceeds the benchmark value, a smaller weight is assigned to the moving average generated by the moving average with a larger input size, and a larger weight is assigned to the moving average generated by the moving average with a smaller input size.

19. The method of claim 16, wherein, The rate of change of the trajectory includes a numerical value representing the speed at which the target object, tracked by the at least one radar sensor, moves in the lateral direction.

20. The method of claim 16, wherein, Based on the determination that the number of the plurality of moving averages is less than a specified number, it is considered that the object is not being tracked by the at least one radar sensor.

Images