Distributed Text Recognition Model Training Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the amount of text data increases, single-machine model training tools for text recognition face memory overflow and training interruption due to limited resources, affecting recognition accuracy.
Innovation Solution
A distributed training process across multiple nodes, where each node obtains recognition coefficients for sample phrases and determines model parameters, generating a recognition model that avoids memory overload and improves training efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a single-machine model training tool is used to generate a recognition model, then the training process is simple to implement, but memory overflow and training interruption occur due to limited memory resources
Solution Approach 1:
The patent divides the training process into multiple independent training tasks that can be executed in parallel across multiple computing devices. Each training task processes a subset of training data independently, allowing the system to handle large datasets without overwhelming single-machine memory resources. This segmentation resolves the contradiction by maintaining operational simplicity through automated task distribution while preventing memory overflow through parallel processing.
Solution Approach 2:
The patent transitions from a single-machine sequential training approach to a multi-device parallel training architecture. By adding the spatial dimension of multiple computing devices, the system can distribute memory loads across different physical machines, thereby preventing memory overflow while maintaining training reliability without significantly increasing operational complexity.
2Measurement precision
If more memory resources are allocated to perform recognition on increasing text data, then recognition accuracy can be improved, but the cost and complexity of the system increases
Solution Approach 1:
The patent combines multiple computing devices into a coordinated parallel training system, where each device contributes its memory resources to the overall training process. This merging approach allows the system to achieve high recognition accuracy by leveraging collective memory capacity across multiple devices without requiring any single device to have excessively large memory, thereby balancing accuracy requirements with system complexity.
3Measurement precision
If a large quantity of text data is used for training, then model accuracy can be improved, but memory overflow occurs due to limited operation resources
Solution Approach 1:
The patent segments the large training dataset into multiple smaller subsets that can be processed in parallel across different training tasks. Each training task loads only its assigned subset into memory, preventing memory overflow while collectively utilizing the entire large dataset for training. This enables the system to achieve high model accuracy through comprehensive data coverage without exceeding individual device memory limits.
Solution Approach 2:
The patent implements continuous parallel training across multiple devices, where training operations proceed simultaneously on different data subsets without interruption. This continuous parallel processing allows the system to efficiently utilize large quantities of training data, maintaining high model accuracy while distributing memory consumption across multiple devices to prevent overflow.
Data Source
AI summary
A text recognition method and apparatus, and a storage medium are provided. The method includes: obtaining sample text data, the sample text data comprising a plurality of sample phrases; and generating a recognition model based on the sample phrases by performing training on a plurality of training nodes. Generating the recognition model includes respectively obtaining, by each of the plurality of training nodes, recognition coefficients of the sample phrases distributed to the corresponding training node; and determining, by the plurality of training nodes, model parameters of the recognition model according to the recognition coefficients of the sample phrases. The method also includes obtaining to-be-recognized text data; inputting the text data to the recognition model; and obtaining recognized target text data output by the recognition model and corresponding to the text data.