Forecast Chart System with Recursive ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning algorithms require manual adjustment by users to create forecast charts, especially when dealing with large datasets, which is inefficient and time-consuming for applications like topic modeling, recommender systems, and internet search queries.
Innovation Solution
A computer-implemented system that connects multiple data sources to a web application, authenticates them, retrieves unstructured data, formats it into structured data using JSON format, and uses Machine Learning models for recursive multi-step forecasting to generate prediction data, which is then displayed as forecast charts on a dashboard.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual adjustment of ML algorithms is used to create forecast charts, then users can customize the analysis, but the process becomes time-consuming and inefficient
Solution Approach 1:
The system performs automatic data processing and ML model training without requiring manual user intervention. The web application automatically retrieves unstructured data from connected data sources, formats it into structured data, trains ML models, and generates forecast charts, allowing the system to serve itself rather than requiring continuous user adjustment.
Solution Approach 2:
The system pre-processes and formats unstructured data into structured data before ML model training, and pre-trains ML models on the structured data and stores them. This preliminary preparation work eliminates the need for manual data preparation and model configuration when users want to create forecast charts, significantly reducing the time required.
2Productivity
If automated ML algorithms are implemented for large datasets, then processing efficiency improves, but system complexity increases
Solution Approach 1:
The system segments the complex data processing workflow into distinct modular components: data retrieval module, data formatting module, ML model training module, and chart generation module. Each module handles a specific task independently, making the overall complex system manageable and maintainable while achieving high processing efficiency on large datasets.
Solution Approach 2:
The web application acts as an intermediary layer between data sources and ML models. It retrieves unstructured data from multiple data sources, formats it into structured data, and prepares it for ML model training. This intermediary processing simplifies the interaction between complex components and enables automated efficient processing of large datasets.
3Adaptability or versatility
If multiple data sources are integrated, then data comprehensiveness improves, but authentication and data retrieval complexity increases
Solution Approach 1:
The web application implements a universal authentication mechanism that can handle multiple data sources with different authentication requirements. The system provides versatile data source connectivity by supporting various authentication methods (API keys, OAuth, etc.) through a unified interface, enabling comprehensive data integration without requiring separate complex authentication processes for each data source.
Data Source
AI summary
A computer-implemented system and method for creating forecast charts. The computer-implemented system includes a computing device, processor, and memory. The processor is configured to connect data sources to a website application. The processors are configured to retrieve, by the website application, unstructured data from the plurality of data sources. The processors are configured to format unstructured data and convert the unstructured data into structured data. The processors are configured to transform, by the website application, the structured data into one or more forecast charts by performing computations by using transformation methods. The structured data is used on dashboard templates, and independent metric templates, wherein at least one of dashboard template is selected from the plurality of dashboard templates presented over a user interface of the computing device by a user. The processors are configured to train Machine Learning (ML) models on the structured data of the dashboard templates. The processors are configured to deploy the ML models on a server. The server is connected to an application programming interface (API). The ML models utilize a recursive multi-step forecasting and adjust a parameter of the recursive multi-step forecasting to fit the structured data. The ML models are used to provide prediction data based on structured data. The processors are configured to create a dashboard based on the dashboard template selected by the user. The API receives and returns the prediction data to the web application where the forecast charts are displayed based on the prediction data.