Neural network model for sequence prediction with attention to entity relationships

A neural network with attention mechanisms prioritizing temporal order over spatial position and using entity-specific embeddings addresses the challenge of capturing cross-entity relationships, enhancing predictive performance and resource efficiency.

WO2026117281A1PCT designated stage Publication Date: 2026-06-04MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 1 Cites 0 Cited by

Patent Information

Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
MICROSOFT TECHNOLOGY LICENSING LLC
Filing Date
2025-07-26
Publication Date
2026-06-04

AI Technical Summary

Technical Problem

Existing neural network models struggle with capturing cross-entity relationships, particularly in long inputs, and require extensive computational resources for domain-specific fine-tuning, while large language models (LLMs) underweight salient information in the middle portions of inputs, leading to degraded predictive performance.

Method used

Implement a neural network with attention mechanisms that prioritize temporal order over spatial position, using entity-specific embeddings and a non-standardized tokenizer to capture cross-entity relationships, enabling a single model to perform both entity matching and ranking, and reduce computational overhead.

Benefits of technology

The neural network effectively identifies and predicts cross-entity relationships in long inputs, improving predictive performance and reducing the need for subsequent fine-tuning, thereby optimizing computational resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2025039382_04062026_PF_FP_ABST
    Figure US2025039382_04062026_PF_FP_ABST
Patent Text Reader

Abstract

An example formulates a training input for a neural network model with attention to include action data and descriptive content. The action data includes a first entity identifier (ID) and a first sequence of actions associated with the first entity ID. The descriptive content describes a first entity associated with the first entity ID. An action in the first sequence of actions includes an electronic transmission involving the first entity and a second entity. An example uses the training input, including the first entity ID, and a non-standardized tokenizer, to train the neural network model with attention to generate and output a second sequence of actions.
Need to check novelty before this filing date? Find Prior Art