Data Augmentation for Detection System Evaluation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current security screening technologies face uncertainty in detecting novel or modified threats due to limited training data and variability in machine responses, lacking effective methods to assess detection performance across different materials and machines.

Innovation Solution

The method involves data augmentation using real data collected from threat or simulant materials, modifying it to emulate responses from various materials or conditions, and analyzing these augmented images with the detection algorithm to evaluate system performance accurately and efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If testing is performed using real threat materials throughout the feature spaces, then detection performance accuracy is improved, but testing complexity and resource requirements increase significantly

Engineering Contradiction:
Improvedetection performance accuracyVSAvoidtesting complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent creates virtual copies of real threat materials through data augmentation techniques. Synthetic threat materials are generated by transforming existing real threat data, allowing comprehensive testing across the entire feature space without physically obtaining and testing every possible threat variant. This copying approach maintains detection performance accuracy while dramatically reducing testing complexity.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent systematically varies parameters of threat materials in synthetic data generation, including material composition, density, size, shape, and spatial arrangement. By changing these parameters across the full feature space, the method achieves comprehensive detection performance evaluation without the exponential resource requirements of physical testing of all parameter combinations.

Inventive Principle:
Principle #35Parameter changes

2Reliability

If multiple machines are used to assess detection performance variation, then machine-to-machine variation is captured, but testing time and resource requirements increase

Engineering Contradiction:
Improvedetection performance variation assessmentVSAvoidtesting time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent creates virtual representations of multiple detection machines and their performance characteristics through synthetic data. By modeling machine-to-machine variation in the synthetic data generation process, the system can assess detection performance variation across virtual machine populations without physically deploying and coordinating multiple actual machines for testing purposes.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent pre-characterizes machine-to-machine variation through initial measurements and incorporates these characteristics into the synthetic data generation model. This preliminary characterization allows subsequent virtual testing to rapidly assess detection performance variation without requiring actual multiple-machine coordination during the testing phase, significantly reducing testing time.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If comprehensive testing with real materials is performed, then detection performance across all threats is evaluated, but the quantity of real materials required increases

Engineering Contradiction:
Improvedetection performance evaluationVSAvoidquantity of real threat materials
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent generates extensive synthetic threat material data by copying and transforming a small set of real threat material measurements. Through data augmentation techniques including geometric transformations, noise addition, and parameter variations, the system creates large volumes of virtual threat data that covers the entire feature space, eliminating the need to procure and test large quantities of actual threat materials.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent develops a universal synthetic data generation framework that can produce diverse threat material representations from a single real threat dataset. This multi-functional approach allows the same real threat measurements to serve as the basis for generating numerous virtual threat variants across different material types, geometries, and configurations, maximizing the evaluation coverage while minimizing real material requirements.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS11068750B1Testing and evaluating detection process by data augmentation
Publication Date: 2021.07.20 THE GOVERNMENT OF THE UNITED STATES OF AMERICA AS REPRESENTED BY THE SEC OF HOMELAND SECURITY
  • US11068750B1 patent drawing
  • US11068750B1 patent drawing
  • US11068750B1 patent drawing

AI summary

In an example, a method includes: based on a starting set of real image data of a set of one or more original images obtained using a detection process of a detection system, identifying elements of the real image data which are picture or volume elements; performing data augmentation on the identified elements to produce one or more augmented images; replacing the set of one or more original images with a set of the one or more augmented images; analyzing the set of one or more augmented images (which may be supplemented with additional real data) using the detection process; and evaluating a detection response of the detection system for each augmented image of the set of one or more augmented images. The analyzing and evaluating may be performed by using an emulator for the detection system which was used to obtain the one or more original images.