Text Processing Acceleration Operator for NLP Frameworks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current deep learning frameworks for natural language processing (NLP) face challenges in optimizing training and reasoning performance, with inconsistent development experiences and high deployment costs in industrial scenarios.

Innovation Solution

The proposed solution involves a deep learning framework that uses a text processing acceleration operator to perform text processing and parallel acceleration, combining this with a transformer encoder operator for natural language understanding and a transformer decoder operator for natural language generating. This framework automatically generates high-performance C++ code from interpreted language processes, enabling multithreading concurrent acceleration and reducing framework scheduling overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If traditional deep learning frameworks are used for text processing, then development is simpler with interpreted languages, but processing speed and training efficiency are slow

Engineering Contradiction:

Improveease of developmentVSAvoidprocessing speed

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent replaces interpreted language execution with compiled C++ code execution. The text processing acceleration operator compiles Python-like pseudocode into optimized C++ code, substituting the interpreted execution model with a compiled execution model that provides both ease of development (through high-level syntax) and high speed (through native compilation).

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the execution parameter from interpreted to compiled by introducing a code generation and compilation mechanism. The text processing acceleration operator translates high-level text processing operations into compiled C++ code, fundamentally changing how the processing is executed while maintaining developer-friendly syntax.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If multiple operators are used for text processing in traditional frameworks, then functionality is comprehensive, but framework scheduling overhead increases and deployment cost rises

Engineering Contradiction:

Improvetext processing functionalityVSAvoidframework scheduling overhead

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent merges multiple text processing operators into a single integrated text processing acceleration operator. This unified operator handles tokenization, word piece segmentation, and other text processing tasks that were previously distributed across multiple operators, eliminating framework scheduling overhead between them while maintaining comprehensive functionality.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The text processing acceleration operator is designed as a universal multi-functional operator that can perform various text processing tasks (tokenization, segmentation, encoding) within a single operator framework. This multi-functionality reduces the need for multiple specialized operators and their associated scheduling overhead.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If parallel processing is implemented in interpreted languages, then concurrency is achieved, but execution speed remains limited by interpretation overhead

Engineering Contradiction:

Improveparallel processing capabilityVSAvoidinterpretation overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent substitutes the interpreted parallel execution model with a compiled parallel execution model. By generating and compiling C++ code for parallel text processing operations, the system eliminates interpretation overhead while maintaining parallel processing capabilities, achieving both high concurrency and high execution speed.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12223271B2Text processing method, device and storage medium

Publication Date: 2025.02.11 BEIJING BAIDU NETCOM SCI & TECH CO LTD

AI summary

Provided are a text processing method, a device and a storage medium, relating to a field of computer technology, and especially to a field of artificial intelligence, such as natural language processing and deep learning. The specific implementation scheme includes: performing text processing on first text, by using a text processing acceleration operator; and processing, in parallel and faster, content after the text processing, by using the text processing acceleration operator. Text processing and parallel acceleration are carried out by the text processing acceleration operator, which can improve the speed of text processing.