AV Cost Function Learning with Pareto Dominance Margins

The patent converts the harmful effect of suboptimal outliers and demonstration noise into a beneficial filtering mechanism. By intentionally designing the cost function to penalize deviations from demonstrated behavior, the system causes outliers to naturally produce higher cost values, allowing them to be identified and filtered out as non-Pareto optimal solutions. This transforms the problem of noise sensitivity into a solution where noise automatically marks itself for rejection.

The patent applies preliminary filtering of demonstrations before final cost function optimization. By pre-processing the demonstration data to identify and remove obvious outliers based on initial cost evaluations, the system prepares cleaner training data that reduces the impact of noise on subsequent optimization. This preliminary action prevents noise from corrupting the final learned cost function.

The patent implements self-service data filtering where the learning system automatically identifies and filters out suboptimal outliers through its own cost function evaluations. The system uses its learned cost function to assess each demonstration, automatically marking suboptimal ones as non-Pareto optimal without requiring external data cleaning intervention. This self-service mechanism eliminates the need for manual or complex automated data filtering pipelines.

The patent incorporates feedback loops where the cost function continuously evaluates demonstrations and adjusts its learned parameters based on Pareto optimality assessments. This feedback mechanism allows the system to automatically adapt to the quality of incoming demonstrations, reinforcing learning from high-quality data while naturally downweighting or rejecting suboptimal outliers through iterative refinement.