Trained Model Construction with Staged Quality Evaluation Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion prediction systems for robots face challenges in evaluating the quality of trained models, as various factors such as user operation quality, training data quality, model quality, and autonomous operation quality are intertwined, making it difficult to assess each aspect individually and leading to inefficient trial and error when improving the model.
Innovation Solution
A methodical construction process for trained models involving six steps: data collection, initial data evaluation and collection refinement, training data selection, training data evaluation and refinement, model construction, and final model evaluation, allowing for step-by-step evaluation and identification of issues at each stage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning is performed using collected data to construct a motion prediction model, then the model can achieve autonomous operation capability, but it becomes difficult to evaluate the quality of each component (user operation, training data, model) individually due to their composite reflection in the final quality
Solution Approach 1:
The patent segments the quality evaluation process into distinct stages: data collection quality evaluation, training data selection quality evaluation, and trained model quality evaluation. Each stage has its own evaluation criteria and can be assessed independently, allowing identification of specific quality issues without the components being composite-reflected as in traditional end-to-end evaluation.
2Productivity
If traditional machine learning construction is used without intermediate evaluation steps, then the process is simpler, but inefficient trial and error is required when improvement is needed
Solution Approach 1:
The patent implements feedback mechanisms at each construction stage. After data collection, quality is evaluated and feedback determines whether to collect more data. After training data selection, quality evaluation provides feedback on whether to select different data. After model training, quality evaluation feedback indicates whether to retrain or adjust. This staged feedback eliminates inefficient trial-and-error by providing directional guidance at each step.
3Measurement precision
If multiple quality aspects are evaluated simultaneously in the final model output, then comprehensive assessment is achieved, but it is unclear what specific aspect needs improvement
Solution Approach 1:
The patent divides comprehensive quality assessment into segmented evaluations at different stages. Instead of one simultaneous comprehensive evaluation, the system performs: (1) data collection quality evaluation, (2) training data quality evaluation, and (3) model performance evaluation. Each evaluation provides clear guidance on what needs improvement at that specific stage, making the improvement direction unambiguous while still achieving comprehensive assessment across all aspects.
Data Source
AI summary
A construction method of a trained model includes six processes. In a first process, data for performing machine learning of an operation of a controlled machine by a human is collected. In a second process, collected data that is the data collected is evaluated and, when it does not satisfy a predetermined evaluation criterion, the data is collected again. In a third process, training data is selected from the collected data that satisfies the evaluation criterion. In a fourth process, the training data is evaluated and, when it does not satisfy a predetermined evaluation criterion, the training data is selected again. In a fifth process, a trained model is constructed by machine learning using the training data that satisfies the evaluation criterion. In a sixth process, the trained model is evaluated and, when it does not satisfy a predetermined evaluation criterion, the trained model is trained again.