Stochastic non-maximum suppression (NMS) method for object recognition, and apparatus for applying same

Abstract

An apparatus for applying a stochastic non-maximum suppression (NMS) method for object recognition is disclosed. A processor of the apparatus can acquire an image for object recognition, determine a bounding box set on the basis of the positional relationship with the bounding box that has the maximum confidence score for object recognition for each class of object in the acquired image, determine a candidate bounding box that has the maximum cumulative distribution function (CDF) by using the determined bounding box set, update the bounding box that has the maximum confidence score on the basis of the determined candidate bounding box, and display the updated bounding box that has the maximum confidence score. By providing the apparatus, an object in an input image can be accurately recognized.

Description

Probabilistic non-maximum suppression (NMS) method for object recognition and device applying the same

[0001] The present disclosure relates to a technique for recognizing objects in an image. More specifically, the present disclosure relates to a probabilistic non-maximum suppression (NMS) method for object recognition and a device applying the same.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.

[0003] Recent advances in deep learning have significantly increased the reliability of multi-class object detection, and it is now being utilized as a core technology in various industries. Early object detection methods, such as Viola Jones (VJ) and Histogram of Oriented Gradient (HOG), were primarily used to detect single objects, such as people or vehicles, rather than multi-class objects. In contrast, deep learning-based object detection methods can simultaneously detect diverse multi-class objects, such as people, vehicles, motorcycles, traffic lights, and traffic signs, with minimal increase in complexity.

[0004] When a typical deep learning-based object detection technique receives an input image from a pre-trained object detection network, it generates numerous bounding boxes around the objects to be detected. Furthermore, typical deep learning-based object detection techniques utilize a non-maximum suppression (NMS) algorithm to remove all other bounding boxes, leaving only one.

[0005] However, in the case of the NMS algorithm, only one bounding box with relatively high reliability accuracy is used through comparison operations and neighboring bounding boxes are not removed and used, so it is bound to be a problem when the reliability accuracy of the bounding box for object recognition is low.

[0006] The problem to be solved by the present disclosure is to provide a method for setting a bounding box more precisely in order to perform object recognition in an input image more accurately.

[0007] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0008] In order to solve the above-described problem, a device applying a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure may include a communication unit; an input unit; a display; a memory storing one or more instructions; and one or more processors executing the instructions stored in the memory.

[0009] The above processor can obtain an image for object recognition through the communication unit or the input unit, and determine a set of bounding boxes based on the positional relationship with the bounding box with the maximum confidence score for object recognition for each class in the obtained image.

[0010] The processor may determine a candidate bounding box having a maximum cumulative distribution function (CDF) using the determined set of bounding boxes, update the bounding box having a maximum confidence score based on the determined candidate bounding boxes, and output the updated bounding box having a maximum confidence score to the acquired image through the display.

[0011] The processor can determine the set of bounding boxes based on bounding boxes whose value of a difference measurement function with respect to the bounding box with the maximum confidence score is greater than or equal to a threshold value.

[0012] The above processor can determine a candidate bounding box for object recognition based on [Formula 1] below.

[0013] [Formula 1]

[0014]

[0015] Here, is a candidate bounding box, i is the index of the bounding box in the set of bounding boxes, a i is the weight of the probability density function (PDF) for the i-th bounding box of the set of bounding boxes, F i ( ) is the probability for the above candidate bounding box, may be a cumulative distribution function (CDF) for the above candidate bounding box, and a candidate bounding box having a maximum value of the cumulative distribution function (CDF) may be determined.

[0016] The above processor can update the bounding box with the maximum confidence score based on [Formula 2] below.

[0017] [Formula 2]

[0018] ,

[0019] Here, is the bounding box with the maximum updated confidence score, λ is the learning rate, can be a candidate bounding box.

[0020] The area of ​​the above candidate bounding box can be configured to be determined based on the area of ​​the bounding box with the maximum confidence score. For example, the area of ​​the bounding box with the maximum confidence score It can be configured to solve [Equation 1] under the condition of a ship.

[0021] In addition, a device applying a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure may include a communication unit; an input unit; a display; a memory storing one or more instructions; and one or more processors executing the instructions stored in the memory.

[0022] The above processor can calculate recognition accuracy for each class through an NMS (non maximum suppression) model from the image for object recognition obtained above, and determine a set of bounding boxes based on a positional relationship with a bounding box having a maximum confidence score for object recognition only for classes in which the calculated recognition accuracy is below a preset standard.

[0023] The processor may determine a candidate bounding box having a maximum cumulative distribution function (CDF) using the determined set of bounding boxes, update the bounding box having a maximum confidence score based on the determined candidate bounding boxes, and output the updated bounding box having a maximum confidence score and the bounding box corresponding to a class whose calculated recognition accuracy exceeds a preset standard together to the acquired image through the display.

[0024] In addition, a probabilistic non-maximum suppression method for object recognition according to the present disclosure may include: obtaining an image for object recognition; determining a set of bounding boxes based on a positional relationship with a bounding box having a maximum confidence score for object recognition for each class in the obtained image; determining a candidate bounding box having a maximum cumulative distribution function (CDF) using the determined set of bounding boxes; updating the bounding box having a maximum confidence score based on the determined candidate bounding boxes; and displaying the updated bounding box having a maximum confidence score on the obtained image.

[0025] The step of determining the set of bounding boxes may include the step of determining the set of bounding boxes based on bounding boxes having a value of a difference measurement function greater than or equal to a threshold value with respect to the bounding box having the maximum confidence score.

[0026] In addition, a computer program stored in a computer-readable recording medium may be further provided to execute a method for implementing the present disclosure.

[0027] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0028] By applying the probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure, objects in acquired images can be recognized more accurately.

[0029] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0030] FIG. 1 is a diagram schematically illustrating a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure.

[0031] FIG. 2 is a block diagram showing the configuration of a device that applies a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure.

[0032] FIG. 3 is a sequence diagram illustrating a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure.

[0033] FIG. 4 is a diagram for explaining a probabilistic non-maximum suppression method for object recognition according to the present disclosure.

[0034] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0035] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0036] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0037] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0038] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0039] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0040] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0041] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0042] The "device applying the probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure" described herein may be implemented as a cloud or cluster system, as well as various devices capable of performing computational processing and providing results to a user. For example, the device applying the probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure may include a computer, a server device, and a portable terminal, or may be in the form of any one of them.

[0043] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0044] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0045] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0046] The artificial intelligence-related functions according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0047] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0048] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0049] The processor can create a neural network, train or learn a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network.

[0050] Before proceeding with a description of Figure 1, it should be noted that the non-maximum suppression (NMS) method is a technique used in computer vision and object detection. It can be used to filter out duplicate bounding boxes generated by object detection algorithms. The NMS method identifies potential objects or regions of interest, and in the process, multiple bounding boxes can be generated, each of which can be assigned a confidence score.

[0051] The NMS method sorts multiple bounding boxes in descending order based on their confidence scores, selects the sorted bounding boxes in descending order, calculates the intersection over union (IoU) between the selected bounding boxes and other bounding boxes in the sorted list, and removes other bounding boxes with high IoU between the selected bounding boxes and other bounding boxes.

[0052] FIG. 1 is a diagram schematically illustrating a probabilistic non-maximum suppression (NMS) method for object recognition according to the present disclosure.

[0053] A device (100 in Fig. 2) applying a probabilistic NMS method for object recognition can recognize an object in an acquired image (S1) for object recognition in two ways.

[0054] For example, a device (100) that applies a probabilistic NMS method for object recognition can first recognize an object based on NMS for an acquired image (S2), and then secondarily recognize the object based on probabilistic NMS to increase recognition accuracy (S3).

[0055] As another example, a device (100) that applies a probabilistic NMS method for object recognition can directly recognize an object based on probabilistic NMS for an acquired image (S3).

[0056] A device (100) applying a probabilistic NMS method for object recognition can perform object recognition by considering the positional relationship of neighboring bounding boxes, rather than simply removing the remaining bounding boxes and leaving only one bounding box for each class based on IoU for object recognition. Accordingly, object recognition can be performed with higher accuracy than when using only one bounding box with high reliability accuracy.

[0057] FIG. 2 is a block diagram showing the configuration of a device (100, hereinafter referred to as “probabilistic NMS method application device”) that applies a probabilistic NMS method for object recognition according to the present disclosure.

[0058] Referring to FIG. 2, the probabilistic NMS method application device (100) may include a communication unit (110), an input unit (120), a display (130), a memory (150), and at least one processor (190). The components of the probabilistic NMS method application device (100) illustrated in FIG. 2 are not essential for implementing the probabilistic NMS method application device (100) according to the present disclosure, and thus the probabilistic NMS method application device (100) described in this specification may have more or fewer components than the components listed above.

[0059] Among the above components, the communication unit (110) may include one or more components that enable communication with various devices equipped with communication modules, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module. The short-range communication module may include a module for recognizing the approach of an external device (e.g., a UWB (Ultra-Wideband) communication module).

[0060] The input unit (120) is for inputting image information (or signal), audio information (or signal), data, or information input from a user, and may include at least one camera, a touch input unit equipped on a touch screen, at least one microphone, and at least one user input interface. Voice data or image data collected by the input unit (120) may be analyzed and processed into a user's control command.

[0061] The output unit is for generating output related to visual, auditory or tactile sensations, and may include at least one of a display (130), an audio output unit, a haptic module and an optical output unit.

[0062] The display (130) can be formed as a touch screen by forming a mutual layer structure with the touch sensor or by forming an integral structure. This touch screen can function as a user input unit that provides an input interface between the device (100) and the user, and at the same time, provide an output interface between the device and the user.

[0063] The display (130) displays (outputs) information processed in the device (100). For example, the display (130) may display execution screen information of an application program (e.g., an application) running in the device, or UI (User Interface) or GUI (Graphical User Interface) information according to such execution screen information.

[0064] The memory (150) can store data supporting various functions of the probabilistic NMS method application device (100), a program for the operation of the processor, can store input / output data (e.g., music files, still images, moving images, etc.), and can store a plurality of application programs (or applications) run by the probabilistic NMS method application device (100), data for the operation of the device (100), and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0065] The memory (150) can store one or more instructions and can store a deep learning-based neural network model.

[0066] The memory (150) may include a storage medium corresponding to at least one type of a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (150) may be a database that is separate from the probabilistic NMS method application device (100) but is connected by wire or wirelessly, and may be implemented as a database system.

[0067] The processor (190) may include one or more processors and may include at least one core. The processor (190) may execute instructions stored in the memory (150). The processor (190) may be implemented with a memory that stores data regarding an algorithm for controlling the operation of components within the probabilistic NMS method application device (100) or a program that reproduces the algorithm, and at least one processor (not shown) that performs the aforementioned operation using the data stored in the memory. In this case, the memory and the processor may be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

[0068] The functions related to artificial intelligence according to the present disclosure may be operated through a processor and memory. The processor (190) may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the memory. Alternatively, when one or more processors are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0069] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights and performs neural network operations through operations between the calculation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated during the learning process so that the loss or cost values ​​obtained from the artificial intelligence model are reduced or minimized. The artificial neural network may include a deep neural network (DNN).

[0070] The probabilistic NMS method application device (100) can provide various UIs in the form of web services based on a platform. For example, it can be provided in the form of a website or web application, but is not limited thereto. In addition, the platform can be provided in the form of a PC application, a mobile application, etc., but the embodiment is not limited thereto. In this case, various user terminals can utilize the various UIs provided by the probabilistic NMS method application device (100) based on a platform.

[0071] At least one component may be added or deleted to correspond to the performance of the components illustrated in Figure 2. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the system.

[0072] Meanwhile, each component illustrated in FIG. 2 refers to software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0073] FIG. 3 is a sequence diagram for explaining object recognition through the probabilistic NMS method performed by the probabilistic NMS method application device (100) according to the present disclosure, and FIG. 4 is a diagram for explaining the probabilistic NMS method according to the present disclosure. FIG. 4 will be referred to together when explaining FIG. 3.

[0074] At step S310, the processor (190) can acquire an image for object recognition through the communication unit (110), the input unit (120), and the memory (150). One or more recognition target objects may be placed in the acquired image, and the placed objects may be one or more for each class (type).

[0075] Referring to FIG. 4, the processor (190) can obtain an image of the inside of a refrigerator containing objects of various classes (types) (e.g., canned peach soda, grapefruit juice, carbonated beverage, etc.).

[0076] In step S320, the processor (190) can determine a set of bounding boxes based on the positional relationship with the bounding box with the maximum confidence score for object recognition for each class in the acquired image.

[0077] Referring to FIG. 4, the processor (190) can set multiple bounding boxes for object recognition for each class (e.g., water, etc.) (before P1 or P3 is performed, left image), and can calculate a confidence score for each of the multiple bounding boxes. For example, the processor (190) can determine a set of bounding boxes based on a positional relationship with the bounding box with the maximum confidence score based on a bounding box with the maximum confidence score of a specific object, and the processor (190) can include one or more bounding boxes whose confidence score is lower than the bounding box with the maximum confidence score in the set of bounding boxes based on the positional relationship with the bounding box with the maximum confidence score.

[0078] In an embodiment, the processor (190) may form a plurality of bounding boxes based on the color, boundary, shape of the object, etc. of the acquired image, and the processor (190) may calculate a confidence score of the plurality of bounding boxes based on the label (bounding box) of each object acquired in advance.

[0079] Here, can be the i-th bounding box for each class, Is It could be the trust score. may be the bounding box with the maximum initial confidence score per class.

[0080] The processor (190) selects the bounding box with the maximum confidence score ( ) can be used to determine the set of bounding boxes based on the bounding boxes whose values ​​of the difference measurement function are greater than or equal to a threshold. For example, here, the threshold may be an intersection over union (IoU) threshold (Г), and the bounding box with the maximum confidence score ( ) can be set to 0.5, 0.6, etc. as a value related to the overlapping area, but the embodiment is not limited thereto.

[0081] The device (100) applying the probabilistic NMS method disclosed in this specification can improve the object recognition rate by applying a method (a method of sequentially applying P1 and P2 or a method of directly applying P3) of using other bounding boxes in addition to the bounding box with the highest confidence score for object recognition, rather than the conventional method (applying only P1) of removing all remaining bounding boxes except the bounding box with the highest confidence score. In particular, by providing the device (100) applying the probabilistic NMS method, a bounding box with a relatively high confidence score can be effectively corrected when its own confidence score is low.

[0082] At step S330, the processor (190) can determine a candidate bounding box having a maximum cumulative distribution function (CDF) value using the determined set of bounding boxes.

[0083] The processor (190) can determine a candidate bounding box for object recognition based on [Formula 1] below.

[0084] [Formula 1]

[0085]

[0086] Here, is a candidate bounding box, i is the index of the bounding box within the set of bounding boxes, which can be the index of the bounding box within the class-specific intersection over union (IoU) threshold (Г), and a i is the weight of the probability density function (PDF) for the i-th bounding box in the set of bounding boxes, and F i ( ) is the probability for the candidate bounding box, can be a cumulative distribution function (CDF) for the candidate bounding boxes for each class. That is, the candidate bounding boxes are It can be a bounding box that maximizes the output (argmax).

[0087] Below, F i ( ) to explain the method for calculating the equation, and a single bounding box, which is one of multiple bounding boxes. Let's explain based on a single bounding box. Assuming that the horizontal and vertical directions are independent of each other, the following [Equation 2] can be derived, and according to the above assumption, the probability can be calculated by dividing horizontally and vertically separately.

[0088] [Formula 2]

[0089]

[0090] Here, The probability density function (PDF) can be modeled by [Equation 3] below.

[0091] [Formula 3]

[0092]

[0093] Here, is the bounding box The reliability accuracy of, is the bounding box The center of, Is width of, Is It can be of height.

[0094] In the embodiment, the bounding box The probability density function (PDF) of can be implemented as an exponential distribution, a uniform distribution, a Pareto distribution, etc. for the convenience of computation. For example, in the case of an exponential distribution, it can be calculated by [Equation 4] below for the convenience of computation, but various probability density functions can also be applied.

[0095] [Formula 4]

[0096]

[0097] In this way, the bounding box Once the probability density function (PDF) is defined, the cumulative distribution function (CDF) can be calculated according to [Equation 5] below.

[0098] [Formula 5]

[0099]

[0100] bounding box If we limit the probability to the domain, the probability can be calculated by [Equation 6] below.

[0101] [Formula 6]

[0102]

[0103] Here, the bounding box , left corner point ( , ) and right corner point ( , ) for bounding boxes The probability for the domain can be calculated.

[0104] The processor (190) can extend the method of calculating CDF for a single bounding box to cases where multiple bounding boxes are applied.

[0105] The processor (190) is, in the above [Formula 1], a for the set of bounding boxes i When the weight of the probability density function (PDF) for the i-th bounding box of the set of bounding boxes is set, A candidate bounding box that maximizes ( ) can be produced.

[0106] Here, the processor (190) has a weight a i can be set to various increasing functions. For example, the processor (190) may be a i It can be set to a polynomial with an increasing slope, an exponential function, etc.

[0107] In this way, when determining a candidate bounding box, the processor (190) can consider the surrounding bounding boxes, thereby overcoming the technical limitation (low accuracy) of the prior art that removes all surrounding bounding boxes and then leaves only the bounding box with the highest confidence score.

[0108] At this time, the area of ​​the candidate bounding box can be configured to be determined based on the area of ​​the bounding box with the maximum confidence score.

[0109] In one embodiment, the area of ​​a candidate bounding box is the area of ​​the bounding box with the maximum confidence score. ship( can be a real number greater than 0) and can have the same area. In this case, the shape of the candidate bounding box can be implemented differently from the bounding box with the maximum confidence score.

[0110] In one embodiment, the processor (190) verifies the object recognition rate by setting the area of ​​the candidate bounding box to be the same as the area of ​​the bounding box with the maximum confidence score, but configuring the shape to be the same or different, and then, if the object recognition rate is determined to be lower than a reference value, the object recognition rate can be tuned by reducing the area of ​​the candidate bounding box.

[0111] In other embodiments, the area and shape of the candidate bounding box may be different from the bounding box with the maximum existing confidence score.

[0112] At step S340, the processor (190) can update the bounding box with the maximum confidence score based on the determined candidate bounding box.

[0113] Specifically, the processor (190) can update the bounding box with the maximum confidence score according to [Formula 7] below.

[0114] [Formula 7]

[0115]

[0116] Here, λ is the learning rate of the algorithm, can be a candidate bounding box, and the existing maximum confidence score The new maximum trust score can be updated.

[0117] At step S350, the processor (190) can display the bounding box with the maximum updated confidence score on the target image for recognition. At this time, the processor (190) can display the maximum bounding box for each class. The processor (190) is similar to the existing NMS method in that it displays only one bounding box for each class, but can ultimately display the bounding box by considering surrounding bounding boxes.

[0118] Referring to FIG. 4, the processor (190) can recognize an object by immediately applying the probabilistic NMS method (P3) after various bounding boxes are generated.

[0119] In addition, the processor (190) may not apply the probabilistic NMS method immediately, but may apply the probabilistic NMS method only to objects with low recognition accuracy (e.g., 410, carbonated beverage) after first applying the NMS method (P1).

[0120] That is, the processor (190) calculates the recognition accuracy for each class through an NMS (non maximum suppression) model from an image for object recognition, and determines a set of bounding boxes based on the positional relationship with the bounding box with the maximum confidence score for object recognition only for classes in which the calculated recognition accuracy is below a preset standard.

[0121] The processor (190) determines a candidate bounding box with a maximum cumulative distribution function (CDF) using the determined set of bounding boxes, updates the bounding box with a maximum confidence score based on the determined candidate bounding boxes, and outputs the bounding box with the maximum updated confidence score and the bounding box corresponding to a class whose calculated recognition accuracy exceeds a preset standard together to an image acquired through the display (130). For example, the processor (190) can increase the confidence accuracy (from 410 to 420) of an object by applying a probabilistic NMS method.

[0122] The processor (190) may apply the NMS method, which is advantageous in terms of computational speed, and then, if a predetermined condition (for example, if the recognition accuracy for a specific object is below a preset standard) is met, apply the probabilistic NMS method only to the object in question.

[0123] Meanwhile, the processor (190) can output a bounding box for each class of each object in the acquired image to the display (130) and then display the confidence accuracy at the top of the bounding box.

[0124] In an embodiment, the processor (190) may first apply a fast NMS method to the acquired image to display the confidence accuracy for each class of each object, and then highlight and output the bounding box of the object below a preset standard (e.g., 0.8 or less).

[0125] In an embodiment, the processor (190) may notify the user via the display (130) that the confidence accuracy of the bounding box of the object is low, and then suggest applying probabilistic NMS to the bounding box of the object. In addition, the processor (190) may also display an estimated time required for applying the probabilistic NMS.

[0126] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0127] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0128] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A device that applies a probabilistic non-maximum suppression (NMS) method for object recognition. Communication section; Input section; Display; Memory that stores one or more instructions; and comprising one or more processors for executing the instructions stored in the memory; The above processor, A device configured to acquire an image for object recognition through the communication unit or the input unit, determine a set of bounding boxes based on a positional relationship with a bounding box having a maximum confidence score for object recognition for each class in the acquired image, determine a candidate bounding box having a maximum cumulative distribution function (CDF) using the determined set of bounding boxes, update the bounding box having a maximum confidence score based on the determined candidate bounding boxes, and output the updated bounding box having a maximum confidence score to the acquired image through the display.

2. In paragraph 1, The above processor, A device configured to determine a set of bounding boxes based on bounding boxes having a value of a difference measurement function greater than or equal to a threshold value with respect to a bounding box having a maximum confidence score.

3. In paragraph 2, The above processor, A device configured to determine a candidate bounding box for object recognition based on [Formula 1] below. [Formula 1] ( : candidate bounding box, i: index of bounding box in the set of bounding boxes, a i : The weight of the probability density function (PDF) for the i-th bounding box in the set of bounding boxes, F i ( ): Probability for the above candidate bounding box, : Cumulative distribution function (CDF) for the above candidate bounding boxes.

4. In paragraph 3, The above processor, A device configured to update the bounding box with the maximum confidence score based on [Formula 2] below. [Formula 2] ( : Bounding box with maximum updated confidence score, λ: learning rate, : candidate bounding box).

5. In paragraph 4, A device configured such that the area of ​​the above candidate bounding box is determined based on the area of ​​the bounding box having the maximum confidence score.

6. A device that applies a probabilistic non-maximum suppression (NMS) method for object recognition. Communication section; Input section; Display; Memory that stores one or more instructions; and comprising one or more processors for executing the instructions stored in the memory; The above processor, A probabilistic non-maximum suppression device configured to calculate recognition accuracy for each class through an NMS (non-maximum suppression) model from the acquired image for object recognition, determine a set of bounding boxes based on a positional relationship with a bounding box having a maximum confidence score for object recognition only for classes in which the calculated recognition accuracy is below a preset standard, determine a candidate bounding box having a maximum cumulative distribution function (CDF) using the determined set of bounding boxes, update the bounding box having a maximum confidence score based on the determined candidate bounding boxes, and output the updated bounding box having a maximum confidence score and the bounding box corresponding to the class in which the calculated recognition accuracy exceeds the preset standard together to the acquired image through the display.

7. As a probabilistic non-maximum suppression method for object recognition, Step of acquiring an image for object recognition; A step of determining a set of bounding boxes based on the positional relationship with the bounding box having the maximum confidence score for object recognition for each class in the acquired image; A step of determining a candidate bounding box having a maximum cumulative distribution function (CDF) using the above-determined set of bounding boxes; A step of updating the bounding box with the maximum confidence score based on the above-determined candidate bounding boxes; and A probabilistic non-maximum suppression method, comprising the step of displaying a bounding box having a maximum updated confidence score on the acquired image.

8. In paragraph 7, The step of determining the set of bounding boxes is: A probabilistic non-maximum suppression method, comprising the step of determining a set of bounding boxes based on bounding boxes having a value of a difference measurement function greater than or equal to a threshold value with respect to the bounding box having the maximum confidence score.

9. In paragraph 8, The step of determining the above candidate bounding box is: A probabilistic non-maximum suppression method for object recognition, comprising a step of determining a candidate bounding box for object recognition based on [Formula 3] below. [Formula 3] ( : candidate bounding box, i: index of bounding box in the set of bounding boxes, a i : The weight of the probability density function (PDF) for the i-th bounding box in the set of bounding boxes, F i ( ): Probability for the above candidate bounding box, : Cumulative distribution function (CDF) for the above candidate bounding boxes.

10. In paragraph 9, The above updating steps are: A probabilistic non-maximum suppression method for object recognition, comprising a step of updating a bounding box having a maximum confidence score, based on [Formula 4] below. [Formula 4] ( : Bounding box with maximum updated confidence score, λ: learning rate, : candidate bounding box).

Images