Semantic Identifier Encoding With Self-Supervised Reconstruction

Solution Overview

Problem

Existing methods for assigning semantic identifiers to documents require a two-step process involving separately trained encoders and semantic indexers, which is time-consuming and resource-intensive.

Innovation Solution

A machine learning model is trained using self-supervised signals to determine semantic identifiers through an encoder-decoder architecture, utilizing embeddings and codebook data to iteratively refine token assignments, reducing the need for separate encoding and indexing steps.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a two-step process with separately trained encoders and semantic indexers is used, then semantic identifier assignment can be performed, but the process becomes time-consuming and resource-intensive

Engineering Contradiction:

Improvesemantic identifier accuracyVSAvoidencoding time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent combines the encoder and semantic indexer into a single integrated model that performs both text encoding and semantic identifier generation in one unified process, eliminating the need for separate training and execution steps while maintaining accuracy through joint optimization of both functions

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The integrated model serves multiple functions simultaneously: it encodes input text, generates semantic identifiers, and can reconstruct the original text from identifiers, making a single system capable of performing tasks that previously required separate specialized components

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Measurement precision

If a two-step process with separately trained encoders and semantic indexers is used, then semantic identifier assignment can be performed, but computational resources are consumed excessively

Engineering Contradiction:

Improvesemantic identifier accuracyVSAvoidcomputational resources

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent combines the encoder and semantic indexer into a single integrated model that performs both text encoding and semantic identifier generation in one unified process, eliminating the need for separate training and execution steps while maintaining accuracy through joint optimization of both functions

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The model uses self-supervised learning where the original text is masked and then recovered through the semantic identifier generation process, using the reconstruction task as a supervisory signal to improve learning efficiency and reduce the need for extensive external training data

Inventive Principle:

Principle #34Discarding and recovering

3Measurement precision

If well-trained machine learning models are used for document encoding and semantic identifier assignment, then accuracy is improved, but the process requires complex separate training procedures

Engineering Contradiction:

Improvesemantic identifier accuracyVSAvoidtraining process complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent combines the encoder and semantic indexer into a single integrated model that performs both text encoding and semantic identifier generation in one unified process, eliminating the need for separate training and execution steps while maintaining accuracy through joint optimization of both functions

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The model uses self-supervised learning where the original text is masked and then recovered through the semantic identifier generation process, using the reconstruction task as a supervisory signal to improve learning efficiency and reduce the need for extensive external training data

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12561529B1Systems for encoding documents and generating semantic identifiers

Publication Date: 2026.02.24 AMAZON TECH INC

AI summary

A semantic identifier that represents information within a document may be determined by training a language model or other machine learning model using a self-supervised process. The model encodes text to generate an embedding. A decoder determines a semantic identifier token that represents information in the text based on the embedding and a previous semantic identifier token if present. The identifier determined by the decoder is aligned with one of a selected set of semantic identifiers indicated in codebook data. To determine the accuracy of the semantic identifier token, a second decoder is used to attempt to reconstruct the original text using the semantic identifier token and a portion of the original text. Differences between the reconstructed text and the original text are used to determine a loss value, and the parameters of the machine learning model or the codebook data are trained based on minimizing the loss value.