Distributed Image Recognition Training with Parallel Sub-Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image recognition technologies face inefficiencies in training image samples, particularly in big data environments, where sequential training of massive data sets leads to low training efficiency and increased time consumption.
Innovation Solution
Implementing a distributed computing framework that divides image sample data sets into sub-blocks and distributes them across computing nodes for parallel training, using algorithms like 2DPCA and LDA to enhance feature extraction and recognition efficiency, and performing incremental training for newly-added samples to reduce retraining of the entire data set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential training of image samples is performed, then training accuracy can be ensured, but training efficiency deteriorates and time consumption increases
Solution Approach 1:
The patent divides the image sample data set into multiple sub-blocks and distributes them to different computing nodes for parallel processing. This segmentation allows simultaneous training of multiple sample subsets, dramatically improving training efficiency while maintaining recognition accuracy through comprehensive coverage of the entire data set.
Solution Approach 2:
The patent combines multiple computing nodes to work together on training different sub-blocks of the image sample data set. By merging computational resources across multiple nodes and aggregating their results, the system achieves both high training efficiency and accurate recognition outcomes.
2Productivity
If distributed computing framework is used for parallel training, then training efficiency improves, but system complexity increases
Solution Approach 1:
The patent segments the training task into independent sub-blocks that can be processed in parallel across multiple computing nodes. This segmentation reduces the computational burden on each individual node and simplifies the overall system architecture by creating modular, independently processable units.
Solution Approach 2:
The patent designs a universal distributed training framework that can handle different types of image sample data and training algorithms through standardized interfaces. This multi-functionality reduces system complexity by providing a unified approach that works across various scenarios without requiring separate specialized systems.
3Measurement precision
If entire data set is retrained when new samples are added, then model accuracy is maintained, but time consumption increases
Solution Approach 1:
The patent performs preliminary distribution of new image samples to computing nodes before full retraining is required. By pre-processing and distributing new samples in advance, the system prepares the data structure and reduces the computational overhead when actual retraining needs to occur, thereby reducing overall time consumption while maintaining accuracy.
Solution Approach 2:
The patent implements incremental training that processes only the necessary portions of new data rather than always retraining the entire data set from scratch. This partial action approach updates the model with new samples efficiently, maintaining recognition accuracy while significantly reducing retraining time compared to complete retraining.
Data Source
AI summary
An image processing method for implementing image recognition using a distributed computing framework is provided. In the method, an image sample data set is obtained by processing circuitry of an image processing apparatus. The image sample data set includes a plurality of image samples. The image sample data set is divided by the processing circuitry into image sample sub-blocks according to a quantity of the image samples in the image sample data set and a quantity of computing nodes of the distributed computing framework. The image sample sub-blocks are distributed by the processing circuitry to the computing nodes. Each of the computing nodes performs image recognition sample training based on the one of the image sample sub-blocks distributed to the respective computing node and obtains a training result corresponding to each image sample in the one of the image sample sub-blocks.