Enhancing long-context reasoning capabilities of machine learning models

The LongReason benchmark addresses the challenge of long-context reasoning in machine learning models by synthesizing diverse tasks from short questions, improving model performance through automated decomposition and expansion of data items, thus enhancing long-context reasoning capabilities.

US20260195640A1Pending Publication Date: 2026-07-09BYTEDANCE TECHNOLOGY LTD
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
US · United States
Patent Type
Applications(United States)
Current Assignee / Owner
BYTEDANCE TECHNOLOGY LTD
Filing Date
2025-01-07
Publication Date
2026-07-09

AI Technical Summary

Technical Problem

Machine learning models struggle with tasks requiring long-context reasoning due to the scarcity of publicly available long-context question-answer data, which is both challenging and time-consuming to create, and existing datasets often limit task diversity and focus on narrow categories.

Method used

A synthetic long-context reasoning benchmark (LongReason) is developed to enhance long-context reasoning capabilities by synthesizing diverse tasks from short questions, using a large language model to decompose and expand short reasoning data into long-context data items, ensuring quality through self-verification processes.

Benefits of technology

LongReason provides controllable context lengths and diverse, realistic tasks without labor-intensive human annotation, enhancing the long-context reasoning abilities of machine learning models across various tasks and context lengths.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US20260195640A1-D00000_ABST
    Figure US20260195640A1-D00000_ABST
Patent Text Reader

Abstract

Each short reasoning data item can be automatically decomposed into a background and an inquiry on the background. A plurality of materials can be automatically generated based on the background. Each of the plurality of materials can indicate a key information point of the background. A long-context background can be automatically constructed by randomly embedding the plurality of materials into a set of irrelevant materials. A plurality of long reasoning data items can be automatically generated by combining the long-context background with the inquiry corresponding to each short reasoning data item.
Need to check novelty before this filing date? Find Prior Art