Automated Image Data Cleansing via Gradient Distribution Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the field of artificial intelligence/deep learning, 3D modeling technology requires extensive training data, which necessitates manual filtering and cleansing, making the process inefficient and labor-intensive.
Innovation Solution
A device and method for data cleansing that includes a transceiver and a processor, which performs global continuity detection on images based on a gradient distribution value to determine whether to cleanse data for a training set, thereby automating the filtering process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual filtering and cleansing is used for training data, then data quality can be maintained, but the process becomes labor-intensive and inefficient
Solution Approach 1:
The system performs automated data cleansing using self-developed algorithms that analyze image continuity and gradient distribution. The processor automatically identifies and filters inappropriate training images without human intervention, allowing the system to serve itself in the data preparation process.
Solution Approach 2:
Manual mechanical filtering operations are replaced with computational algorithms that calculate continuous values and gradient distribution values. The processor uses mathematical computations to detect image characteristics and automatically determine whether images should be included in the training set, substituting human manual inspection with automated computational analysis.
2Reliability
If extensive manual filtering is performed on training data, then data quality improves, but time consumption increases significantly
Solution Approach 1:
The system performs preliminary automated analysis of images by calculating continuous values and gradient distribution values before final selection. This preliminary computational action filters out obviously unsuitable images early in the process, preventing time-wasting manual review of clearly inappropriate candidates while maintaining thorough quality control for borderline cases.
Solution Approach 2:
The system transforms the data cleansing process from subjective manual evaluation to objective parameter-based analysis. By calculating specific parameters such as continuous values and gradient distribution values, the system converts quality assessment into measurable quantitative metrics that can be automatically compared against thresholds, enabling rapid and reliable decision-making.
3Productivity
If automated detection methods are implemented, then processing speed increases, but system complexity increases
Solution Approach 1:
The automated detection system is divided into distinct functional modules: a continuous value calculation module that computes image continuity metrics, and a gradient distribution analysis module that evaluates gradient patterns. This segmentation allows each module to specialize in specific computational tasks, improving processing efficiency while organizing system complexity into manageable, independent components.
Data Source
AI summary
A device and a method for data cleansing are provided. The method includes following steps: receiving, by the processor, an image through the transceiver, wherein the image include a picture; when the processor determines that a continuous value corresponding to the image is greater than a continuous value threshold, performing, by the processor, a global continuity detection on the picture to obtain a gradient distribution value corresponding to the picture; and using, by the processor, the gradient distribution value to determine whether to perform a data cleansing corresponding to a training set on the picture.