AI Data Structure Manipulation via Candidate Function Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing platforms struggle with manipulating large datasets due to the need for significant user expertise in both the platform's functionality and programming languages, leading to time-consuming and potentially erroneous custom functions.
Innovation Solution
A computer-implemented method using two artificial intelligence models to generate and validate candidate functions for transforming datasets, where the first AI model generates candidate functions based on a target outcome, and the second AI model provides an AI transformation of a dataset subset, allowing for validation and execution of validated functions on the entire dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users generate customised computing functions to manipulate large datasets, then the functionality for data manipulation is improved, but the time consumption and error risk increase significantly
Solution Approach 1:
The system pre-generates multiple candidate computing functions using AI models before the user needs them. When a user requests a custom function, the system has already prepared a pool of candidate functions that can be quickly evaluated and selected, eliminating the need for users to spend time writing and debugging code from scratch.
Solution Approach 2:
The system automatically generates, validates, and selects computing functions without requiring user programming expertise. The AI models self-service the function generation task by taking user requirements and automatically producing validated computing functions, freeing users from the time-consuming process of manual function creation.
2Adaptability or versatility
If users generate customised computing functions, then specific data manipulation needs are met, but the reliability of the manipulation decreases due to undetected errors
Solution Approach 1:
The system implements a feedback mechanism where candidate computing functions are automatically validated by executing them on test datasets and comparing results against expected outputs. This validation feedback loop identifies and filters out erroneous functions before they are deployed, ensuring only reliable functions are used for actual data manipulation tasks.
Solution Approach 2:
The system performs preliminary validation and error detection on candidate functions before they are applied to the main dataset. By proactively identifying and eliminating erroneous functions in advance, the system prevents errors from propagating to the final results, thereby maintaining high reliability in data manipulation operations.
3Measurement precision
If AI models generate and validate multiple candidate functions, then the accuracy of data transformation is improved, but the computational overhead increases
Solution Approach 1:
The system generates and validates multiple candidate functions (excessive action) to ensure high accuracy, but only applies the necessary number of validations required to achieve sufficient confidence. This balanced approach ensures transformation accuracy while avoiding unnecessary computational waste from excessive validation of already-sufficient candidates.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, apparatuses and computer programs for manipulating data structures are provided. A method includes: receiving an indication of a target outcome, where the target outcome defines a transformation of a first dataset; providing to a first artificial intelligence model one or more requests to generate candidate functions; receiving, in response to the one or more requests, a plurality of distinct candidate functions from the first artificial intelligence model; providing, to a second artificial intelligence model, a request to generate the transformation on a first subset; receiving, from the second artificial intelligence model, an AI transformation of the first subset; executing each of the plurality of candidate functions on the first subset to generate a plurality of test; validating one or more of the candidate functions; and executing one of the validated candidate functions on the first dataset to achieve the transformation of the first dataset.