Neural Network Object Batching for Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current batch scheduling in deep learning technologies is inefficient when handling input data of varying sizes, leading to high resource overhead due to the need for padding and invalid calculations.
Innovation Solution
An object processing method that determines a target grouping combination consuming the minimum resources by batching target objects and inputting them into a neural network model for processing, optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If input data of different sizes is packaged into one batch for processing, then throughput is improved, but resource overhead increases due to padding and invalid calculations
Solution Approach 1:
The patent segments input data of different sizes into multiple batches based on size thresholds. Instead of packaging all data into a single batch with padding, the system divides data into size-based groups (e.g., small, medium, large batches) and processes them separately. This eliminates the need for excessive padding while maintaining high throughput by充分利用 batch processing benefits for each size category.
2Productivity
If batch processing is used to improve calculation efficiency, then throughput increases, but resource consumption increases due to processing padded data
Solution Approach 1:
The patent dynamically changes batch processing parameters (batch size, grouping thresholds) based on input data characteristics. By adjusting these parameters according to actual data size distributions, the system optimizes the balance between calculation efficiency and resource consumption, avoiding fixed batch size limitations while minimizing unnecessary processing of padded data.
3Ease of operation
If simple batch scheduling is used for ease of operation, then implementation is simple, but resource utilization is suboptimal
Solution Approach 1:
The patent introduces dynamic batch scheduling that automatically adjusts grouping strategies based on real-time data characteristics. The system monitors input data size distributions and dynamically modifies batch formation rules, transitioning between different scheduling modes as needed. This maintains operational simplicity while significantly improving resource utilization compared to static scheduling approaches.
Data Source
AI summary
This application discloses an object processing method and apparatus, a computer readable storage medium, and an electronic device. The method includes: grouping objects in an object set to obtain a plurality of grouping combinations; determining, from the plurality of grouping combinations, a target grouping combination that consumes a minimum resource; a resource consumed by each grouping combination in the plurality of grouping combinations comprising: a sum of resources consumed by inputting each group in the each grouping combination to a neural network model for processing; and inputting in batches each group in the target grouping combination to the neural network model for processing.