Method for generating object detection result and electronic apparatus therefor

Abstract

A method for generating a detection result for an object is provided by an electronic device. The method for generating a detection result may include: determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying the object through the object prediction box; setting first threshold values ​​for the physical characteristics for each object class included in the object classes based on a first data set; extracting the physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values; and generating a first detection result for the plurality of objects by removing the first outlier prediction box from the plurality of object prediction boxes.

Description

Technology Field

[0001] The present disclosure relates to a method for generating object detection results and an electronic device for the same, and more specifically, to a method for generating object detection results and an electronic device for the same by setting threshold values ​​for physical characteristics of object prediction boxes presumed to correspond to objects, and filtering out abnormal prediction boxes by comparing physical characteristics extracted from object prediction boxes generated corresponding to objects with the threshold values. Background Technology

[0002] In order for autonomous vehicles to drive safely, it is required to perceive the surrounding environment and accurately detect various objects on the road. To this end, deep learning-based object detection models trained to infer the location, size, and class of objects based on sensors such as LiDAR (Light Detection and Ranging), cameras, and radar, as well as data acquired from these sensors, can be utilized.

[0003] Meanwhile, in the actual driving environment of autonomous vehicles, sensor data regarding unknown objects that are not included in the training data of the object detection model may be input into the model, and false detections may occur, such as when an unknown object is incorrectly classified as a known object. To address this, methods such as detecting unknown objects by utilizing the confidence score output from the object detection model or additionally training the object detection model by including training data regarding unknown objects have been previously proposed; however, these methods have issues such as somewhat low detection accuracy or increased computational costs due to additional training procedures. Furthermore, conventional unknown object detection methods have limitations in that they are not applicable to LiDAR-based 3D object detection, as they have primarily focused on object detection in 2D RGB images. The problem to be solved

[0004] One objective of the embodiments of the present disclosure is to provide a method for generating object detection results and an electronic device for such results, which can improve the accuracy and reliability of object detection results by identifying anomaly prediction boxes based on the physical characteristics of object prediction boxes generated corresponding to objects and generating detection results in which the anomaly prediction boxes are filtered.

[0005] The technical problems to be solved by the present disclosure are not limited to the technical problems described above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. means of solving the problem

[0006] A method for generating a detection result performed by an electronic device according to one embodiment may include: determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box; setting first threshold values ​​for physical characteristics for each object class included in the object classes based on a first data set; extracting physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values; and generating a first detection result for a plurality of objects by removing the first outlier prediction box from the plurality of object prediction boxes.

[0007] In a method for generating a detection result performed by an electronic device according to one embodiment, the step of identifying a first anomaly prediction box may include: determining an object class corresponding to an object prediction box included in a plurality of object prediction boxes; identifying specific first threshold values ​​among first threshold values ​​that are set corresponding to the object class; and identifying an object prediction box as a first anomaly prediction box based on the result of comparing physical characteristics extracted for the object prediction box with specific first threshold values.

[0008] In a method for generating a detection result performed by an electronic device according to one embodiment, the step of determining an object class may include: a step of verifying a class probability vector provided corresponding to an object prediction box; and a step of determining an object class based on an element having a maximum value in the class probability vector.

[0009] In a method for generating detection results performed by an electronic device according to one embodiment, the step of setting first thresholds may include: grouping reference object boxes included in a first data set according to each object class; extracting physical characteristics from the grouped reference object boxes and generating a statistical distribution corresponding to each of the physical characteristics extracted from the grouped reference object boxes; and setting first thresholds for each object class based on the statistical distribution.

[0010] In a method for generating detection results performed by an electronic device according to one embodiment, the first threshold values ​​may include a maximum value and a minimum value identified for each of the physical characteristics extracted from reference object boxes grouped for each object class based on a statistical distribution.

[0011] A method for generating a detection result performed by an electronic device according to one embodiment may further include: a step of setting second threshold values ​​for physical characteristics for each object class based on a second data set; a step of identifying a second abnormal prediction box among a plurality of object prediction boxes based on physical characteristics extracted for each of a plurality of object prediction boxes and the second threshold values; and a step of generating a second detection result for a plurality of objects by removing the second abnormal prediction box from a first detection result, and the second data set may include a greater number of reference object boxes than the first data set.

[0012] In a method for generating detection results performed by an electronic device according to one embodiment, a plurality of object prediction boxes are provided to the electronic device by an object detection model, and the object detection model can be trained to generate a plurality of object prediction boxes by receiving point cloud data generated from a LiDAR sensor as input.

[0013] In a method for generating a detection result performed by an electronic device according to one embodiment, physical characteristics for an object prediction box may include at least one of the area, height, and aspect ratio of the object prediction box.

[0014] A non-transient computer-readable storage medium capable of being readable by storing a program for executing a detection result generation method according to one embodiment on a computer, wherein the detection result generation method comprises: determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box; setting first threshold values ​​for physical characteristics for each object class included in the object classes based on a first data set; extracting physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values; and generating a first detection result for a plurality of objects by removing the first outlier prediction box from the plurality of object prediction boxes.

[0015] An electronic device for performing a detection result generation method according to one embodiment may be provided, comprising: a processor; and one or more memories for storing one or more instructions, wherein, when the one or more instructions are executed, the processor controls the processor to perform the following steps: determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box; setting first threshold values ​​for physical characteristics for each object class included in the object classes based on a first data set; extracting physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values; and generating a first detection result for a plurality of objects by removing the first outlier prediction box from the plurality of object prediction boxes. Effects of the invention

[0016] A method for generating object detection results according to one embodiment of the present disclosure has a technical effect in that it can reduce false positives in real time by extracting physical characteristics of object prediction boxes generated corresponding to objects, identifying anomaly prediction boxes by comparing the extracted physical characteristics with threshold values, and removing the identified anomaly prediction boxes.

[0017] In addition, the object detection result generation method according to one embodiment of the present disclosure has a technical effect in that it effectively improves the detection accuracy of unidentified objects without additional training of the object detection model by setting threshold values ​​for filtering abnormal prediction boxes by considering the physical characteristics of object prediction boxes corresponding to each object class.

[0018] The effects obtainable from the present disclosure are not limited to those described above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below to which the present disclosure pertains. Brief explanation of the drawing

[0019] FIG. 1 is a diagram showing the configuration of an electronic device according to one embodiment. FIG. 2 shows a flowchart of a method in which an electronic device according to one embodiment generates a detection result for an object. FIG. 3 shows a block diagram of an object detection system according to one embodiment. FIGS. 4a to 4d show examples of object prediction boxes generated in an object detection model according to one embodiment. FIG. 5 shows a flowchart of a method for an electronic device according to one embodiment to set threshold values ​​for physical characteristics. FIG. 6 shows a flowchart of a method for an electronic device to identify an anomaly prediction box according to one embodiment. FIG. 7 shows a flowchart of a method for an electronic device to determine an object class according to one embodiment. FIG. 8 shows a flowchart of a method in which an electronic device according to one embodiment generates a detection result for an object. Specific details for implementing the invention

[0020] Hereinafter, various embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice them. The detailed description disclosed below is intended to explain exemplary embodiments of various examples and is not intended to limit the scope to specific embodiments.

[0021] The terms used in the embodiments have been selected to be as widely used as possible, taking into account their functions in the present disclosure; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section. Therefore, terms used in the present disclosure should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0022] The following embodiments are combinations of the components and features of various embodiments in a predetermined form. Each component or feature may be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, various embodiments may be constructed by combining some components and features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

[0023] In the description of the drawings, procedures or steps that could obscure the essence of the various embodiments were not described, nor were procedures or steps that could be understood by a person of ordinary knowledge in the relevant technical field described.

[0024] Throughout the specification, when a part is described as "comprising" or "including" a component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. In this document, the singular form of a noun corresponding to an item may be used to include both singular and plural forms, unless otherwise indicated in the specification or clearly contradicted by the context. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or any combination thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from other corresponding components and do not limit the components in any other aspect (e.g., importance or order).

[0025] Each component (e.g., module or program) of the components described in this document may include a singular or multiple entities. According to various embodiments, one or more of the components or operations may be omitted, or one or more other components or operations may be added. Additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components in the same or similar manner as they were performed by each of the components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, or repeatedly; one or more of the operations may be executed in a different order; may be omitted; or one or more other operations may be added.

[0026] As used in this document, the terms "module" or "...part" refer to a unit that processes at least one function or operation, and may include a unit implemented by hardware, software, firmware, or a combination thereof.

[0027] Various embodiments of this document may be implemented as software (e.g., a program or application) comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine. For example, the processor of the machine may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to at least one called instruction. One or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain a signal, and does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0028] In addition, specific terms used in various embodiments are provided to aid in understanding the various embodiments, and the use of such specific terms may be modified in other forms within the scope of not departing from the technical concept of the various embodiments.

[0029] The detection result generation method and electronic device for the same proposed in the present disclosure may comprise a method for generating detection results for multiple objects and an electronic device for the same, wherein physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box are determined, threshold values ​​for physical characteristics for each object class included in the object classes are set based on a dataset, physical characteristics are extracted for each of the multiple object prediction boxes generated corresponding to multiple objects, and an outlier prediction box among the multiple object prediction boxes is identified based on the physical characteristics extracted for each of the multiple object prediction boxes and threshold values, and the outlier prediction box is removed from the multiple object prediction boxes.

[0030] In particular, the detection result generation method of the present disclosure can be effectively applied to the defense sector, where accurate detection of various objects existing in a battlefield environment is required. For example, military vehicles in the defense sector may be equipped with an object detection system to detect obstacles or enemy targets along their driving paths, and the object detection system may be required to accurately distinguish between confirmed and unconfirmed objects to minimize false positives. In response to such characteristics of the defense sector, the detection result generation method presented in the present disclosure below can be understood as a technical concept easily applicable to the defense sector, in that it generates more reliable object detection results by setting threshold values ​​for the physical characteristics of object prediction boxes for each object class and removing abnormal prediction boxes based on the result of comparing the physical characteristics of multiple object prediction boxes generated corresponding to multiple objects with the threshold values.

[0031] FIG. 1 is a diagram showing the configuration of an electronic device according to one embodiment.

[0032] Referring to FIG. 1, the electronic device (100) may include a processor (110) and a memory (120) according to one embodiment. The electronic device (100) illustrated in FIG. 1 is illustrated only with components related to the present embodiment, and it will be understood by those skilled in the art related to the present embodiment that other general-purpose components may be included in addition to the components illustrated in FIG. 1.

[0033] For example, an electronic device (100) may include a communication device comprising one or more transceivers, an input unit, and an output unit. The communication device is a device for performing wired / wireless communication and can communicate with an external electronic device. The external electronic device may be a terminal or a server. In addition, communication technologies used by the communication device may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc. The input unit may be, for example, a traditional type of keypad or keyboard, a mouse, a microphone for receiving voice signals, a camera, and various other types of input means for detecting or receiving various forms of user input. The output unit may be, for example, a display that outputs images, a speaker that outputs sound, a haptic device that generates vibrations, and various other forms of output means.

[0034] Additionally, at least some of the components within the electronic device (100) may be implemented as an integrated unit or as a single or multiple unit. At least some of the components within the electronic device (100) may be connected to each other via a bus, GPIO (general purpose input / output), SPI (serial peripheral interface) or MIPI (mobile industry processor interface), etc., to exchange data and / or signals.

[0035] The electronic device (100) of FIG. 1 can generate detection results for objects. Specifically, the electronic device (100) can determine physical characteristics for object prediction boxes presumed to correspond to objects and object classes for classifying objects, and can set first threshold values ​​for physical characteristics for each object class included in the object classes based on a first data set. In addition, the electronic device (100) can extract physical characteristics for each of the multiple object prediction boxes generated corresponding to multiple objects, and identify a first outlier prediction box among the multiple object prediction boxes based on the extracted physical characteristics and first threshold values. Furthermore, the electronic device (100) can generate a first detection result for multiple objects by removing the first outlier prediction box from the multiple object prediction boxes.

[0036] A processor (110) is a component capable of performing operations or data processing regarding the control and / or communication of each component of an electronic device (100). For example, the processor (110) can control the electronic device (100) overall by executing programs stored in memory (120) within the electronic device (100). The processor (110) may be implemented as a CPU (central processing unit), GPU (graphics processing unit), AP (application processor), etc., provided within the electronic device (100), but is not limited thereto. Unless there are special circumstances, in this disclosure, the processor (110) may refer to a set of one or more processors (110).

[0037] The processor (110) may be implemented as a computer or a similar device according to hardware, software, or a combination thereof. In terms of hardware, the processor (110) may be implemented in the form of an electronic circuit that processes electrical signals to perform control functions, and in terms of software, it may be implemented in the form of a program that drives the hardware processor (110). Meanwhile, unless otherwise specifically mentioned in the description below, the operation of the electronic device (100) may be interpreted as being performed by the control of the processor (110). That is, when modules implemented in the electronic device (100) are executed to generate detection results for an object, it may be interpreted that the processor (110) controls the modules to perform the following operations of the electronic device (100).

[0038] Memory (120) is hardware that stores various data processed within the electronic device (100), and memory (120) can temporarily or semi-permanently store data processed and data to be processed in the electronic device (100). For example, data regarding an operating system (OS) for driving the electronic device (100) may be stored in the memory (120) of the electronic device (100). Memory (120) may include volatile memory such as DRAM (dynamic random access memory) and SRAM (static random access memory) and / or non-volatile memory such as EEPROM (electrically erasable programmable read-only memory), CD-ROM (compact disc read-only memory), Blu-ray disc, HDD (hard disk drive), and SSD (solid state drive). Such memory (120) may be provided as an embedded type or a removable type. Additionally, memory (120) may store one or more instructions for the operation of the processor (110). Unless there are special circumstances, in the present disclosure, memory (120) may mean a set of one or more memories (120).

[0039] The method for generating a detection result of the present disclosure, performed by the electronic device (100) of FIG. 1, may also be implemented by a computer-readable, non-transient computer-readable storage medium (or non-transient recording medium) for the operation. The method for generating a detection result may be implemented as a software module or algorithm and may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on a processor (110). Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, so that computer-readable code may be stored and executed in a distributed manner. The medium may be computer-readable, stored in memory (120), and executed on a processor (110).

[0040] An electronic device (100) according to one embodiment may further include a display (not shown). The display may visually provide various information to the outside under the control of a processor (110). The display may include a touch circuit configured to detect a user's touch or a sensor circuit configured to measure the intensity of the force generated by the touch.

[0041] FIG. 2 shows a flowchart of a method in which an electronic device according to one embodiment generates a detection result for an object.

[0042] In step S210, an electronic device (hereinafter referred to as the "electronic device") (100) that generates a detection result for an object according to one embodiment may determine physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying the object through the object prediction box. In the present disclosure, an object prediction box may refer to a three-dimensional box representing an area presumed to have an object by an object detection model, and may also be referred to as a bounding box. In one embodiment, the electronic device (100) may determine at least one of the area, height, and aspect ratio of the object prediction box as a physical characteristic for the object prediction box. Here, the area may refer to the area viewed from the BEV (bird's eye view) direction, and the aspect ratio may refer to the ratio of the width to the length of the object prediction box.

[0043] Additionally, in the present disclosure, an object class may refer to a category of an object classified by an electronic device (100) based on an object prediction box, and for example, 'car', 'truck', 'bus', 'pedestrian', etc. may each correspond to a single object class, but the examples of object classes in the various embodiments of the present disclosure are not limited thereto. For example, the object classes determined by the electronic device (100) may include an 'unknown' class corresponding to an unidentified object, and the 'unknown' class may be referred to as an 'ignore' class or an 'OOD (Out-Of-Distribution)' class.

[0044] In step S220, the electronic device (100) can set first threshold values ​​for physical characteristics for each object class included in the object classes based on the first data set. Here, the first data set may refer to a data set containing multiple reference object boxes corresponding to multiple classes, and the reference object boxes may refer to ground truth (GT) boxes generated corresponding to actual objects. In one embodiment, a Nuscenes mini data set, which is a sample of a portion of the Nuscenes data set, may be used as the first data set. The electronic device (100) can group the reference object boxes included in the first data set according to each object class and can generate a statistical distribution corresponding to each of the physical characteristics extracted from the grouped reference object boxes. In one embodiment, the electronic device (100) can identify a maximum value and a minimum value for each of the physical characteristics extracted from the reference object boxes grouped for each object class based on the statistical distribution, which can be expressed as shown in [Table 1] below.

[0045] [Table 1]

[0046]

[0047] [Table 1] shows the maximum and minimum values ​​of area, height, and aspect ratio extracted from reference object boxes corresponding to each of the five object classes, 'car', 'truck', 'bus', 'pedestrian', and 'unknown', in one embodiment. Here, 'unknown' may refer to an object class corresponding to an object not included in 'car', 'truck', 'bus', and 'pedestrian', for example, objects such as police cars, ambulances, carts, and strollers may be included in the 'unknown' class. However, the examples of object classes and physical characteristics in this disclosure are not limited to those described in [Table 1], for example, 'trailer', 'construction vehicle', 'bicycle', 'motorcycle', 'traffic cone', and 'barrier' may be included as object classes, and width and length may be included as physical characteristics. In one embodiment, the electronic device (100) may set first threshold values ​​including a maximum value and a minimum value identified for each of the physical characteristics extracted from grouped reference object boxes. For example, the electronic device (100) may set the first threshold values ​​to the values ​​listed in [Table 1] corresponding to each object class.

[0048] In step S230, the electronic device (100) can extract physical characteristics for each of the plurality of object prediction boxes generated corresponding to the plurality of objects. Specifically, the electronic device (100) can extract physical characteristics determined in step S210 for each of the plurality of object prediction boxes provided by the object detection model. For example, the electronic device (100) can extract area, height, and / or aspect ratio from each of the plurality of object prediction boxes.

[0049] In step S240, the electronic device (100) can identify a first abnormal prediction box among a plurality of object prediction boxes based on physical characteristics extracted for each of the plurality of object prediction boxes and first threshold values. Specifically, the electronic device (100) determines an object class corresponding to an object prediction box included in the plurality of prediction boxes, checks specific first threshold values ​​set corresponding to the determined object class among the first threshold values ​​set in step S220, and then identifies whether the object prediction box corresponds to a first abnormal prediction box based on the result of comparing the physical characteristics extracted from the object prediction box with the specific first threshold values ​​in step S230. At this time, the object class can be determined based on a class probability vector provided by an object detection model corresponding to the object prediction box.

[0050] In one embodiment, the electronic device (100) may define a normal range for each of the physical characteristics of an object prediction box based on specific first threshold values, and may identify the object prediction box as a first abnormal prediction box if at least one of the physical characteristics extracted from the object prediction box falls outside the normal range. A specific embodiment of how the electronic device (100) identifies the first abnormal prediction box will be described in detail with reference to FIG. 6, which will be described later.

[0051] In step S250, the electronic device (100) can generate a first detection result for a plurality of objects by removing a first abnormal prediction box from a plurality of object prediction boxes. That is, the electronic device (100) can generate a first object detection result by filtering a first abnormal prediction box in which physical characteristics deviate from the normal range among a plurality of object prediction boxes provided by an object detection model, and can prevent false detections caused by the first abnormal prediction box. For example, the electronic device (100) can reduce the false positive (FP) caused by the object prediction box by identifying an object prediction box in which the object class is determined to be 'car' class but physical characteristics deviate from the normal range defined for the 'car' class as a first abnormal prediction box and filtering it, thereby improving the average precision (AP). According to one embodiment, as the first abnormal prediction box is filtered, the detection accuracy for unidentified objects can be improved, and detection performance indicators such as AUROC, AUPR-S, FRR95, etc., can be improved.

[0052] In one embodiment, the electronic device (100) can update thresholds for physical characteristics and generate object detection results based on the updated thresholds. Specifically, the electronic device (100) can set second thresholds for physical characteristics for each object class based on a second data set, identify a second anomaly prediction box among a plurality of object prediction boxes based on physical characteristics extracted for each of the plurality of object prediction boxes and the second thresholds, and then generate a second detection result for a plurality of objects by removing the second anomaly prediction box from the first detection result generated in step S250. Here, the second data set may include a larger number of reference object boxes than the aforementioned first data set. That is, the electronic device (100) can update thresholds based on a data set containing more GT boxes and then filter the anomaly prediction box, and accordingly, the accuracy of the object detection result generated by the electronic device (100) can be improved.

[0053] FIG. 3 shows a block diagram of an object detection system according to one embodiment. As illustrated in FIG. 3, the object detection system (300) may include a LiDAR sensor (310), an object detection model (320), and an electronic device (330). In this case, the electronic device (330) may correspond to the electronic device (100) of FIG. 1. Notwithstanding FIG. 3, in one embodiment, the LiDAR sensor (310) and the object detection model (320) may correspond to modules included in the electronic device (330). FIG. 3 illustrates exemplary configurations of the object detection system (300), but the embodiments of the present disclosure are not limited thereto, and, for example, other components may be added in addition to the components illustrated in FIG. 3, or a plurality of components may be integrated into one to be implemented.

[0054] The LiDAR sensor (310) emits a laser pulse into the surrounding environment and can acquire spatial information of an object, such as x, y, and z coordinates, by measuring the time taken for the laser pulse to be reflected from the surface of the object and return to the LiDAR sensor (310), i.e., the Time of Flight (ToF). In one embodiment, the point cloud data (311) may include a timestamp indicating the time when the spatial information was acquired.

[0055] The object detection model (320) can generate object prediction boxes (321) based on point cloud data (311) provided from the LiDAR sensor (310). For example, the object detection model (320) can generate object prediction boxes (321) by extracting features from the point cloud data (311) and predicting object candidate regions where objects are presumed to exist based on the extracted features. Additionally, the object detection model (320) can generate class probability vectors (322) based on the point cloud data (311). In this case, the class probability vectors (322) can correspond to each object prediction box (321), and each class probability vector (322) can represent the probability that the corresponding object prediction box is included in each object class. For example, the class probability vector can be expressed as [car: 0.8, truck: 0.1, bus: 0.05, pedestrian: 0.05]. In one embodiment, the CenterPoint model of the MMDetection3D framework can be used as an object detection model (320).

[0056] The electronic device (330) can generate a detection result (331) based on object prediction boxes (321) and class probability vectors (322) provided by the object detection model (320). Specifically, the electronic device (330) can extract physical characteristics, such as area, height, and aspect ratio, from each of the object prediction boxes (321). Additionally, the electronic device (330) can determine an object class corresponding to each of the object prediction boxes (321) based on elements having maximum values ​​in the class probability vectors (322). For example, if the class probability vector provided corresponding to a specific object prediction box is [car: 0.8, truck: 0.1, bus: 0.05, pedestrian: 0.05], the electronic device (330) can determine the object class of the specific object prediction box as the 'car' class. Furthermore, the electronic device (330) can determine whether there is an abnormal prediction box among the object prediction boxes (321) by checking specific threshold values ​​corresponding to the object class determined for each object prediction box (321) and comparing the physical characteristics of each object prediction box (321) with the specific threshold values. If a specific object prediction box is determined to be an abnormal prediction box, the electronic device (330) can generate a detection result (331) by removing the specific object prediction box among the object prediction boxes (321).

[0057] FIGS. 4a through 4d illustrate examples of object prediction boxes generated in an object detection model according to one embodiment. For example, FIG. 4a may be understood to illustrate a first object prediction box (400a) generated corresponding to a passenger car object, FIG. 4b an object prediction box (400b) generated corresponding to a truck object, FIG. 4c an object prediction box (400c) generated corresponding to a bus object, and FIG. 4d an object prediction box (400d) generated corresponding to a pedestrian object. In one embodiment, the first to fourth object prediction boxes (400a through 400d) may be provided to the electronic device (100) of FIG. 1 after being generated in the object detection model.

[0058] The electronic device (100) can set threshold values ​​for physical characteristics, such as area, height, and aspect ratio, for each object class based on a data set containing reference object boxes. In one embodiment, the electronic device (100) can set threshold values ​​including the identified maximum and minimum values ​​for each of the physical characteristics extracted from the reference object boxes grouped for each object class, as described above with reference to [Table 1]. Additionally, the electronic device (100) can set the range between the identified maximum and minimum values ​​for each of the physical characteristics as a normal range and determine whether each of the first to fourth object prediction boxes (400a to 400d) corresponds to an abnormal prediction box by checking whether the physical characteristics extracted from the first to fourth object prediction boxes (400a to 400d) fall within the normal range. Hereinafter, FIGS. 4a to 4d will be described with reference to [Table 1].

[0059] FIG. 4a illustrates a first object prediction box (400a) generated in an object detection model corresponding to a passenger car object. An electronic device (100) can receive the first object prediction box (400a) from the object detection model and extract at least one physical characteristic. For example, the electronic device (100) can extract a first length (410a), a first width (420a), and a first height (430a) from the object prediction box (400a). In one embodiment, the electronic device (100) can calculate the area and aspect ratio of the first object prediction box (400a) based on the first length (410a) and the first width (420a). For example, the electronic device (100) can calculate the area of ​​the first object prediction box (400a) by multiplying the first length (410a) and the first width (420a), and can calculate the aspect ratio of the first object prediction box (400a) by dividing the first length (410a) by the first width (420a).

[0060] Additionally, the electronic device (100) can determine whether the first object prediction box (400a) corresponds to an abnormal prediction box based on the area, height, and aspect ratio of the first object prediction box (400a). For example, if the area of ​​the first object prediction box (400a) is 14 , when the height is calculated to be 2.1m and the aspect ratio to be 2.2, the electronic device (100) determines that the first object prediction box (400a) corresponds to a normal prediction box based on [Table 1] and can generate a detection result including the first object prediction box (400a). Meanwhile, the area of ​​the first object prediction box (400a) is 17 When calculated as such, the electronic device (100) can identify the first object prediction box (400a) as an abnormal prediction box and can generate a detection result in which the first object prediction box (400a) is removed.

[0061] FIG. 4b illustrates a second object prediction box (400b) generated in an object detection model corresponding to a truck object. An electronic device (100) can receive the second object prediction box (400b) from the object detection model and extract at least one physical characteristic. For example, the electronic device (100) can extract a second length (410b), a second width (420b), and a second height (430b) from the object prediction box (400b). In one embodiment, the electronic device (100) can calculate the area and aspect ratio of the second object prediction box (400b) based on the second length (410b) and the second width (420b). For example, the electronic device (100) can calculate the area of ​​the second object prediction box (400b) by multiplying the second length (410b) and the second width (420b), and can calculate the aspect ratio of the second object prediction box (400b) by dividing the second length (410b) by the second width (420b).

[0062] Additionally, the electronic device (100) can determine whether the second object prediction box (400b) corresponds to an abnormal prediction box based on the area, height, and aspect ratio of the second object prediction box (400b). For example, if the area of ​​the second object prediction box (400b) is 15 If the height is calculated to be 2.8m and the aspect ratio to be 3.7, the electronic device (100) can determine that the second object prediction box (400b) corresponds to a normal prediction box based on [Table 1] and can generate a detection result including the second object prediction box (400b). Meanwhile, if the aspect ratio of the second object prediction box (400b) is calculated to be 2.5, the electronic device (100) can identify the second object prediction box (400b) as an abnormal prediction box and can generate a detection result with the second object prediction box (400b) removed.

[0063] Similarly, the electronic device (100) can extract a third length (410c), a third width (420c), and a third height (430c) from a third object prediction box (400c) generated in an object detection model corresponding to a bus object, and calculate the area and aspect ratio of the third object prediction box (400c) based on the third length (410c) and the third width (420c). Afterward, the electronic device (100) can determine whether the third object prediction box (400c) corresponds to an abnormal prediction box based on [Table 1], and if the third object prediction box (400c) is determined to be an abnormal prediction box, it can generate a detection result in which the third object prediction box (400c) is removed.

[0064] Likewise, the electronic device (100) can extract a fourth length (410d), a fourth width (420d), and a fourth height (430d) from a fourth object prediction box (400d) generated in an object detection model corresponding to a pedestrian object, and calculate the area and aspect ratio of the fourth object prediction box (400d) based on the fourth length (410d) and the fourth width (420d). Subsequently, the electronic device (100) can determine whether the fourth object prediction box (400d) corresponds to an abnormal prediction box based on [Table 1], and if the fourth object prediction box (400d) is determined to be an abnormal prediction box, it can generate a detection result in which the fourth object prediction box (400d) is removed.

[0065] FIG. 5 shows a flowchart of a method for an electronic device to set threshold values ​​for physical characteristics according to one embodiment. As described above in step S220, the electronic device (100) can set first threshold values ​​for physical characteristics for each object class based on a first data set including reference object boxes.

[0066] In step S510, the electronic device (100) can group reference object boxes included in the first data set according to each object class. For example, the first data set may include a plurality of reference object boxes labeled with corresponding object class information, and the electronic device (100) can group the plurality of reference object boxes based on the labeled object class information. In one embodiment, the electronic device (100) can perform grouping by classifying the plurality of reference object boxes into one of 'car', 'truck', 'bus', 'pedestrian', and 'unknown' as shown in [Table 1].

[0067] In step S520, the electronic device (100) can extract physical characteristics from grouped reference object boxes and generate a statistical distribution corresponding to each of the physical characteristics extracted from the grouped reference object boxes. Specifically, the electronic device (100) can extract width, length, and height from the grouped reference object boxes in step S510 and calculate an area and aspect ratio based on the extracted width and length. Additionally, the electronic device (100) can generate a statistical distribution corresponding to each of the area, height, and aspect ratio.

[0068] In step S530, the electronic device (100) may set first threshold values ​​for each object class based on a statistical distribution. Specifically, the electronic device (100) may set first threshold values ​​for each object class based on at least one metric identified from the statistical distribution of physical characteristics generated in step S520. For example, the electronic device (100) may set the maximum and minimum values ​​of each physical characteristic identified based on the statistical distribution as the first threshold values, as described above with reference to [Table 1], but the embodiments of the present disclosure are not limited thereto, and in one embodiment, the electronic device (100) may set the first threshold values ​​based on the mean and standard deviation of each physical characteristic identified from the statistical distribution, and each of the physical characteristics Values ​​corresponding to specific percentiles, such as 10% and 90%, may also be set as the first threshold values.

[0069] FIG. 6 shows a flowchart of a method for an electronic device to identify an anomaly prediction box according to one embodiment.

[0070] In step S610, the electronic device (100) can determine an object class corresponding to an object prediction box included in a plurality of object prediction boxes. Specifically, the electronic device (100) can receive a class probability vector corresponding to an object prediction box from an object detection model and can determine an object class corresponding to an object prediction box based on the class probability vector. For example, the electronic device (100) can determine the object class of the object prediction box as 'car' class based on the class probability vector.

[0071] In step S620, the electronic device (100) can identify specific first threshold values ​​set corresponding to the object class determined in step S610 among the first threshold values. For example, the electronic device (100) can identify the maximum / minimum area, maximum / minimum height, and maximum / minimum aspect ratio corresponding to the 'car' class as specific first threshold values ​​by referring to [Table 1].

[0072] In step S630, the electronic device (100) can identify a first anomaly prediction box based on the result of comparing physical characteristics extracted from the object prediction box with specific first threshold values. Specifically, the electronic device (100) can determine whether the object prediction box corresponds to a first anomaly prediction box by comparing the physical characteristics extracted for the object prediction box in step S230 with specific first threshold values ​​identified in step S620.

[0073] In one embodiment, the electronic device (100) may set a normal range for each of the physical characteristics based on specific first threshold values ​​identified in step S620, and may identify the object prediction box as a first abnormal prediction box if at least one of the physical characteristics extracted for the object prediction box in step S230 does not fall within the normal range. For example, the electronic device (100) may set the normal area range of the 'car' class to '5.59 to 15.95' by referring to [Table 1]. The normal height range can be set to '1.28 to 3m', the normal aspect ratio range to '2.23 to 2.71', and the extracted area for the object prediction box is '7.8 When the height is '1.8m' and the aspect ratio is '2.3', the object prediction box can be determined as a normal prediction box, whereas the area extracted for the object prediction box is '7.8 When the height is '1.8m' and the aspect ratio is '3.4', the aspect ratio extracted from the object prediction box falls outside the normal aspect ratio range, so the object prediction box can be identified as a first or more than one prediction box.

[0074] The aforementioned steps S610 to S630 can be performed for each of the plurality of object prediction boxes obtained from the object detection model, and accordingly, at least one object prediction box can be identified as a first or more prediction box.

[0075] FIG. 7 shows a flowchart of a method for an electronic device to determine an object class according to one embodiment. As described above in step S610, the electronic device (100) can determine an object class based on a class probability vector provided corresponding to an object prediction box.

[0076] In step S710, the electronic device (100) can check the class probability vector provided in correspondence with the object prediction box. At this time, the class probability vector may include elements having probability values ​​corresponding to each of the object classes. For example, if 'car', 'truck', 'bus', and 'pedestrian' are included in the object classes determined by the electronic device (100), the class probability vector may be expressed as [car: 0.8, truck: 0.1, bus: 0.05, pedestrian: 0.05].

[0077] In step S720, the electronic device (100) can determine an object class based on the element having the maximum value in the class probability vector. For example, if the class probability vector is provided as [car: 0.8, truck: 0.1, bus: 0.05, pedestrian: 0.05], the electronic device (100) can determine the object class corresponding to the object prediction box as the 'car' class.

[0078] In one embodiment, the electronic device (100) may determine the object class as 'unknown' if the maximum value identified in the class probability vector is less than a preset probability threshold. For example, if the class probability vector is provided as [car: 0.4, truck: 0.3, bus: 0.25, pedestrian: 0.05] and the probability threshold is set to 0.7, the electronic device (100) may determine that the object prediction box does not correspond to any of the object classes among 'car', 'truck', 'bus', and 'pedestrian' and determine the object class as 'unknown'. However, the embodiments of the present disclosure are not limited thereto, and for example, the electronic device (100) may determine whether the object prediction box corresponds to the 'unknown' class by applying temperature scaling to the class probability vector based on the ODIN (Out-of-Distribution detector for Neural networks) technique and then analyzing the class probability vector.

[0079] FIG. 8 shows a flowchart of a method in which an electronic device according to one embodiment generates a detection result for an object. In one embodiment, steps S810 to S830 of FIG. 8 may be performed following step S250 of FIG. 2.

[0080] In step S810, the electronic device (100) may set second threshold values ​​for physical characteristics for each object class based on a second data set. Here, the second data set may include a greater number of reference object boxes than the first data set described in step S220. Specifically, the electronic device (100) may group the reference object boxes included in the second data set according to each object class and extract physical characteristics, such as area, height, and aspect ratio, from the grouped reference object boxes. Additionally, the electronic device (100) may generate a statistical distribution corresponding to each of the extracted physical characteristics and set second threshold values ​​for each object class based on the statistical distribution. For example, the electronic device (100) may set values ​​corresponding to specific percentiles, such as 20% and 80%, for each of the physical characteristics identified based on the statistical distribution as the second threshold values.

[0081] In one embodiment, the electronic device (100) may set specific values ​​included in a normal range defined based on first thresholds in step S240 as second thresholds. For example, if the electronic device (100) sets the normal height range of the 'bus' class to '2.81 to 4.58 m' by referring to [Table 1] in step S240, the electronic device (100) may set values ​​corresponding to greater than 2.81 m and less than 4.58 m as second thresholds for the height of the 'bus' class. That is, the second thresholds can be set more precisely by reflecting the physical characteristics of a larger number of GT boxes compared to the first thresholds, and the reliability of the second thresholds as a criterion for filtering anomaly prediction boxes can be improved.

[0082] In step S820, the electronic device (100) can identify a second abnormal prediction box among a plurality of object prediction boxes based on physical characteristics extracted for each of the plurality of object boxes and second threshold values. Specifically, the electronic device (100) determines an object class corresponding to an object prediction box included in the plurality of object boxes, checks specific second threshold values ​​set corresponding to the determined object class among the second threshold values, and then, in step S230, determines the object prediction box as a normal prediction box or a second abnormal prediction box based on the result of comparing the physical characteristics extracted for the object prediction box with the specific second threshold values. In one embodiment, the electronic device (100) can set a normal range for each of the physical characteristics based on specific second threshold values, and if at least one of the physical characteristics extracted for the object prediction box does not fall within the normal range, the object prediction box can be identified as a second abnormal prediction box.

[0083] In step S830, the electronic device (100) can generate a second detection result for a plurality of objects by removing a second anomaly prediction box from the first detection result. Specifically, the electronic device (100) can generate a second detection result for a plurality of objects by filtering the second anomaly prediction box identified in step S820 from the first detection result generated in step S250.

[0084] Through the aforementioned steps S810 to S830, the electronic device (100) can improve the accuracy and reliability of the object detection result by updating thresholds for physical characteristics and filtering anomaly prediction boxes identified based on the updated thresholds.

[0085] In the foregoing, although all components constituting the embodiments disclosed in this document have been described as being combined or operating in combination, the embodiments disclosed in this document are not necessarily limited to such embodiments. That is, within the scope of the purposes of the embodiments disclosed in this document, all components may be selectively combined in one or more ways to operate.

[0086] Furthermore, terms such as "include," "compose," or "have" as described above, unless specifically stated otherwise, mean that the relevant component may be inherent; thus, they should be interpreted as allowing for the inclusion of additional components rather than excluding them. All terms, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments disclosed in this document pertain, unless otherwise defined. Commonly used terms, such as those defined in advance, should be interpreted in accordance with their meaning in the context of the relevant technology and, unless explicitly defined in this document, should not be interpreted in an ideal or overly formal sense.

[0087] The foregoing description is merely an illustrative explanation of the technical concept disclosed in this document, and a person skilled in the art to which the embodiments disclosed in this document pertain can make various modifications and variations within the scope of the essential characteristics of the embodiments disclosed in this document. Accordingly, the embodiments disclosed in this document are intended to explain, not limit, the technical concept of the embodiments disclosed in this document, and the scope of the technical concept disclosed in this document is not limited by these embodiments. The scope of protection of the technical concept disclosed in this document shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this document.

Claims

Claim 1 A method for an electronic device to generate a detection result for an object comprises: determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying the object through the object prediction box; setting first threshold values ​​for the physical characteristics for each object class included in the object classes based on a first data set; extracting the physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values; generating a first detection result for the plurality of objects by removing the first outlier prediction box from the plurality of object prediction boxes; setting second threshold values ​​for the physical characteristics for each object class based on a second data set; and identifying a second outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the second threshold values. A method for generating a detection result, comprising the step of generating a second detection result for the plurality of objects by removing the second anomaly prediction box from the first detection result, wherein the second data set includes a greater number of reference object boxes than the first data set. Claim 2 A method for generating a detection result according to claim 1, wherein the step of identifying the first anomaly prediction box comprises: determining an object class corresponding to an object prediction box included in the plurality of object prediction boxes; identifying specific first threshold values ​​among the first threshold values ​​that are set corresponding to the object class; and identifying the object prediction box as the first anomaly prediction box based on the result of comparing physical characteristics extracted for the object prediction box with the specific first threshold values. Claim 3 A method for generating a detection result according to claim 2, wherein the step of determining the object class comprises: a step of verifying a class probability vector provided corresponding to the object prediction box; and a step of determining the object class based on an element having a maximum value in the class probability vector. Claim 4 A method for generating detection results according to claim 1, wherein the step of setting the first threshold values ​​comprises: grouping reference object boxes included in the first data set according to each object class; extracting physical characteristics from the grouped reference object boxes and generating a statistical distribution corresponding to each of the physical characteristics extracted from the grouped reference object boxes; and setting the first threshold values ​​for each object class based on the statistical distribution. Claim 5 A method for generating detection results according to claim 4, wherein the first threshold values ​​include a maximum value and a minimum value identified for each of the physical characteristics extracted from the grouped reference object boxes for each object class based on the statistical distribution. Claim 6 delete Claim 7 A method for generating a detection result according to claim 1, wherein the plurality of object prediction boxes are provided to the electronic device by an object detection model, and the object detection model is trained to generate the plurality of object prediction boxes by receiving point cloud data generated from a LiDAR sensor. Claim 8 A method for generating a detection result according to claim 1, wherein the physical characteristics for the object prediction box include at least one of the area, height, and aspect ratio of the object prediction box. Claim 9 In a computer-readable, non-transient computer-readable storage medium storing a program for executing a detection result generation method on a computer, the detection result generation method comprises: a step of determining physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box; a step of setting first threshold values ​​for the physical characteristics for each object class included in the object classes based on a first data set; a step of extracting the physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; and a step of identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values. A non-transient computer-readable storage medium comprising: a step of generating a first detection result for the plurality of objects by removing the first anomaly prediction box from the plurality of object prediction boxes; a step of setting second threshold values ​​for physical characteristics for each object class based on a second data set; a step of identifying a second anomaly prediction box among the plurality of object prediction boxes based on physical characteristics extracted for each of the plurality of object prediction boxes and the second threshold values; and a step of generating a second detection result for the plurality of objects by removing the second anomaly prediction box from the first detection result, wherein the second data set includes a greater number of reference object boxes than the first data set. Claim 10 An electronic device for performing a method for generating detection results comprises: a processor; and one or more memories for storing one or more instructions, wherein, when the one or more instructions are executed, the processor determines physical characteristics for an object prediction box presumed to correspond to an object and object classes for classifying an object through the object prediction box; a step of setting first threshold values ​​for the physical characteristics for each object class included in the object classes based on a first data set; a step of extracting the physical characteristics for each of a plurality of object prediction boxes generated corresponding to a plurality of objects; and a step of identifying a first outlier prediction box among the plurality of object prediction boxes based on the physical characteristics extracted for each of the plurality of object prediction boxes and the first threshold values. An electronic device characterized by controlling a processor to perform the steps of: generating a first detection result for the plurality of objects by removing the first anomaly prediction box from the plurality of object prediction boxes; setting second threshold values ​​for physical characteristics for each object class based on a second data set; identifying a second anomaly prediction box among the plurality of object prediction boxes based on physical characteristics extracted for each of the plurality of object prediction boxes and the second threshold values; and generating a second detection result for the plurality of objects by removing the second anomaly prediction box from the first detection result, wherein the second data set includes a greater number of reference object boxes than the first data set.

Images