Soft Label Generation for Data Node Clustering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The challenge lies in efficiently labeling large quantities of data obtained by data nodes, which is costly and resource-intensive when performed by human annotators, and often requires more computing resources than the data nodes possess.

Innovation Solution

A data node manager generates and assigns soft labels using processed data from data nodes, merging and clustering this data to create soft label metadata, which includes probabilistic label distributions, and uses this metadata for labeling actions such as assigning hard labels or providing labels to human annotators.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If human annotators perform labeling, then labeling accuracy is maintained, but cost and resource consumption increase significantly

Engineering Contradiction:

Improvelabeling accuracyVSAvoidresource consumption

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system enables data nodes to perform self-labeling by generating soft labels through clustering algorithms using their own processed data, eliminating the need for external human annotators and significantly reducing resource consumption while maintaining labeling quality

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical process of human annotation with automated computational clustering algorithms that process and label data through mathematical operations, achieving both efficiency and accuracy without human intervention

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Quantity of substance

If data nodes perform labeling locally, then resource usage is optimized, but computing resources required exceed what data nodes possess

Engineering Contradiction:

Improveresource usage efficiencyVSAvoidcomputing resource capacity

Core Design Contradiction:

Quantity of substanceVSPower

Solution Approach 1:

The labeling task is segmented into multiple stages: data nodes perform initial data processing and feature extraction locally, then aggregate results to a manager node for clustering-based soft label generation, distributing computational workload across the network rather than requiring each node to have full capabilities

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A manager node acts as an intermediary between data nodes and the clustering process, collecting processed data from multiple nodes and performing the computationally intensive clustering operations centrally, then distributing soft labels back to individual nodes for final labeling actions

Inventive Principle:

Principle #24Intermediary (Mediator)

3Quantity of substance

If automated labeling is implemented, then resource consumption is reduced, but labeling complexity increases

Engineering Contradiction:

Improveresource consumptionVSAvoidlabeling process complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The system transforms the labeling problem from a binary classification task into a soft labeling problem using probability distributions as parameters, where data points are assigned probabilistic labels through clustering algorithms, simplifying the automated process while reducing resource requirements compared to traditional human annotation

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11762895B2Method and system for generating and assigning soft labels for data node data

Publication Date: 2023.09.19 EMC IP HLDG CO LLC

AI summary

Techniques described herein relate to a method for managing data of data nodes. The method includes obtaining, by a data node manager, a soft labeling request; in response to obtaining the soft labeling request: sending, by the data node manager, requests for processed data to data nodes associated with the data node manager; obtaining, by the data node manager, processed data from the data nodes; merging, by the data node manager, the processed data to obtain processed data; performing, by the data node manager, clustering on the processed data to obtain soft label metadata; associating, by the data node manager, the soft label metadata with live data associated with the data nodes; and performing, by the data node manager, labeling actions using the live data and the soft label metadata.