Language-Based Object Detection Using Negative Descriptions and Images

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing object detection systems struggle to effectively utilize negative labels in natural language-based object detection, leading to limitations in discriminative model performance and semantic understanding.

Innovation Solution

The method involves generating negative descriptions and images using language models and generative image models, which are then used to train object detection models, thereby expanding the label space and improving discriminative loss.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If natural language is used to describe object semantics, then the label space size increases, but the system's ability to effectively utilize negative labels deteriorates

Engineering Contradiction:
Improvelabel space sizeVSAvoidnegative label utilization
Core Design Contradiction:
Quantity of substanceVSReliability

Solution Approach 1:

The patent segments the label space into positive labels (objects present in the image) and negative labels (objects absent from the image). This segmentation allows the system to independently process and utilize negative labels, resolving the contradiction by organizing the expanded label space into distinct functional categories that can be effectively managed despite the overall increase in label quantity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent inverts the traditional object detection approach by explicitly generating and processing negative labels alongside positive labels. Instead of only detecting what is present, the system generates descriptions of what is absent and uses these negative labels for training, thereby improving negative label utilization while maintaining the expanded label space capability.

Inventive Principle:
Principle #13The other way round (Inversion)

2Adaptability or versatility

If generative models are used to create negative images, then data augmentation capability improves, but computational complexity increases

Engineering Contradiction:
Improvedata augmentation capabilityVSAvoidcomputational complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by pre-generating negative images and their corresponding negative labels before the actual object detection training process. This allows the computationally intensive generative modeling to be performed in advance, creating a ready-to-use augmented dataset that can then be efficiently utilized during training without repeatedly invoking complex generative models.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by generating negative images that are modifications or transformations of existing positive images rather than creating entirely new images from scratch. By copying the structure and content of original images and selectively modifying them to represent absent objects, the system reduces computational complexity while maintaining data augmentation effectiveness.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS20250118096A1Language-based object detection and data augmentation
Publication Date: 2025.04.10 NEC LABORATORIES AMERICA INC
  • US20250118096A1 patent drawing
  • US20250118096A1 patent drawing
  • US20250118096A1 patent drawing

AI summary

Methods and systems for object detection include generating a negative description for an input image based on a positive description of the input image using a language model. A negative image is generated based on the input image and the negative description by replacing a portion of the input image that is described by the positive description with content that is described by the negative description using a generative image model. An object detection model is trained with the input image, the positive description, the negative description, and the negative image.