3-d object detection and classification from imagery

Abstract

A system and method for recognizing objects in an image is described. The system can receive an image from a sensor and detect one or more objects in the image. The system can further detect one or more components of each detected object. Subsequently, the system can create a segmentation map based on the components detected for each detected object and determine whether the segmentation map matches a plurality of 3-D models (or projections thereof). Additionally, the system can display a notification through a user interface indicating whether the segmentation map matches at least one of the plurality of 3-D models.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]The present application is a continuation application of U.S. Ser. No. 17 / 001,779 which application was filed on Aug. 25, 2020 and will issue as U.S. Pat. No. 11,138,410 on Oct. 5, 2021.FIELD OF THE INVENTION[0002]The present invention pertains to object detection and processing.BACKGROUND AND SUMMARY[0003]Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of objects of a certain class (e.g., humans, buildings, or cars) in digital images and videos. Methods for object detection generally fall into either classical machine learning-based approaches or deep learning-based approaches. For Classical Machine Learning approaches, it becomes necessary to first define features, then using a technique such as support vector machine (SVM) to do the classification.[0004]3D object recognition involves recognizing and determining 3D information, such as the pose, volume, or sha...

AI Technical Summary

Benefits of technology

[0007]It is an object of this disclosure to describe a target recognition model and system which can accurately detect targets when no data from those targets can be collected. Additionally, it is an object of this disclosure to provide for a target recognition model and system which can enable a user to evaluate and analyze the model's output after the model makes one or more predictions.

[0009]In contrast to standard end-to-end deep learning networks, the proposed invention separates the target detection and classification process into a multi-stage process. The target recognition model disclosed herein can offer a new approach to computer aided object detection and classification. This model can leverage generic, robust background information (e.g., what wheels look like in a vehicle) and known component relationships (i.e., the size and shape of components on the vehicle) to perform reliable, explainable target detection and classification. This model can solve the robustness problem by training a multi-task, multi-purpose data processing backbone which enables robust object detection and component part segmentation. The outputs of this part segmentation network can be then integrated into a component-based target identifier that can be updated using only physical models of new target classes, or 3-D models (e.g., CAD Models). This enables the model to detect new types of objects without requiring any sensor data for these new classes.

[0010]The target recognition technique of the present disclosure is more precise than other target recognition techniques. One of the reasons for this precision is that this technique uses various steps which optimize the target recognition process. For example, many target recognition techniques attempt to directly classify an object. However, in one embodiment, the present disclosure provides for detecting an entity of interest (or broad object class) first, which is much easier and precise than classification of objects. Similarly, the present disclosure provides for segmentation of parts of a detected object (or target), which is also much easier than classification of objects. As such, the final classification of the object based on the segmented parts is easier and more accurate.

Problems solved by technology

Additionally, the output from these models is usually limited to a box drawn around the detected object.

As such, these models do not provide any information about how or why the detected object was recognized or classified.

In fact, oftentimes, there is no data available for these objects.

This lack of data impedes the training process for the model, and thus, reduces the accuracy of the model's predictions.

Furthermore, in these applications, due to the serious consequences that can come about as a result of the prediction, the end-user must be able to immediately interpret and / or verify the system's output, e.g., the end-user must be able to understand how and why the system came up with its conclusion.

Images