Machine Learning Label Correction for Accurate Picking Motion Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for setting a region of interest in video data for behavior analysis, such as manual setting and semantic segmentation, are time-consuming and prone to human errors, leading to inaccurate detection of buying behaviors in retail environments.

Innovation Solution

A computer-readable recording medium and information processing apparatus that utilize machine learning models to automatically set and correct labels for regions of interest by analyzing movement trajectories and direction variations, enhancing the accuracy of behavior analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual setting or semantic segmentation is used to set regions of interest, then the process is simple to understand and implement, but it is time-consuming and prone to human errors

Engineering Contradiction:

Improveaccuracy of region of interest detectionVSAvoidtime required for setting regions of interest

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs automatic label correction by having the behavior analysis process itself identify and correct labeling errors. The behavior recognition unit detects picking motions and uses this information to automatically correct labels in the region of interest extraction process, eliminating the need for manual verification and reducing time loss while maintaining high accuracy.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements a feedback mechanism where the behavior recognition results are fed back into the label correction process. The behavior recognition unit detects actual picking motions, and this information is used to correct the labels set by the first machine learning model, creating a closed-loop system that continuously improves accuracy while automating the process.

Inventive Principle:

Principle #23Feedback

2Reliability

If manual setting is used to define regions of interest, then flexibility in defining custom regions is achieved, but human errors and inconsistency occur

Engineering Contradiction:

Improveconsistency of region labelingVSAvoidcomplexity of automatic label correction system

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system introduces a behavior recognition unit as an intermediary between the region of interest extraction and the final analysis. This intermediary detects actual picking motions and provides correction information to the label correction unit, serving as a mediator that bridges the gap between automatic extraction and manual verification while ensuring consistency and reliability.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system replaces the manual mechanical process of region setting with an automated machine learning-based system. The first machine learning model automatically extracts regions of interest, and the behavior recognition unit automatically detects and corrects labeling errors, substituting human manual operations with automated computational processes that eliminate human errors and ensure consistency.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Productivity

If semantic segmentation is used for region extraction, then automation is achieved, but accuracy deteriorates due to excess or deficiency in region extraction

Engineering Contradiction:

Improvespeed of region extractionVSAvoidaccuracy of behavior detection

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system initially extracts regions of interest using semantic segmentation, which may include some excess regions. Then, the behavior recognition unit detects actual picking motions to identify the precise regions where behaviors occur. The label correction unit uses this information to refine the initially extracted regions, removing excess parts and ensuring accuracy while maintaining the productivity benefit of automatic extraction.

Inventive Principle:

Principle #16Partial or excessive action

4Area of stationary object

If the first machine learning model sets labels for all areas, then comprehensive coverage is achieved, but errors in label setting increase

Engineering Contradiction:

Improvecoverage of labeled areasVSAvoidaccuracy of label setting

Core Design Contradiction:

Area of stationary objectVSMeasurement precision

Solution Approach 1:

The system extracts and corrects only the specific labels that contain errors, rather than reprocessing all labels. The label correction unit identifies areas where the first machine learning model made errors and focuses correction efforts on those specific regions, maintaining comprehensive coverage while improving accuracy by concentrating resources on problematic areas.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12367664B2Computer-readable recording medium storing label change program, label change method, and information processing apparatus

Publication Date: 2025.07.22 FUJITSU LTD

AI summary

A non-transitory computer-readable recording medium stores a label change program for causing a computer to execute a process including: acquiring image data that includes a plurality of areas; setting a label for each of the plurality of areas by inputting the image data to a first machine learning model; specifying a behavior performed by a person located in a first area among the plurality of areas for an object located in a second area; and changing a label set for the second area based on a specified behavior of the person.