Hybrid cybernetic system for forecasting from diverse inputs

Abstract

A hybrid cybernetic system for demand and financial forecasting accepting a variety of input data types is described in the present disclosure. In some embodiments, a process including vector autoregression analysis, neural network prediction, and genetic algorithms is employed to improve the accuracy and transparency of traditional neural network forecasting. In the present disclosure, an additional pre-processing module is included to process a diverse set of numerical and non-numerical input types.

Description

1 Hybrid Cybernetic System for Forecasting from Diverse Inputs Background This present invention relates to a cybernetic system to forecast interdependent dynamics from a variety of numerical and non-numerical data types, including evenly spaced time series data, non-evenly spaced time series data, and qualitative data. Accurately forecasting demand is essential for the effective allocation of resources, such as in the forecasting of tourism demand, which allows for the preparation of food, transportation, hotel rooms, and airline seats. In finance, econometric tests including linear regression, vector autoregression, and other methods have been employed to quantify the relationships between global stock markets. Published literature points to neural networks (NN) as a superior method for forecasting demand in interdependent systems, such as in forecasting stock market dynamics. Nevertheless, hesitancy around adopting neural networks, although well known for their capabilities in pattern recognition, for business forecasting persists. Neural networks draw criticisms for their lack of transparency in producing outputs, including their inability to identify the relevance of and relationship between variables, often acting like a “black box”. A previously published scientific article by S. I. Ao (2010) outlines a hybrid model combining vector autoregression (VAR) to identify the correlated and leading indicators suitable for NN input, neural network model trained by shallow backpropagation to create forecasts based on those inputs, and a genetic algorithm (GA) to adjust the weightings of each NN model and cope with the time-dependent nature of the variable relationships. This system has been shown to perform better than NN systems alone in two case studies: forecasting tourism demand and quantifying cross-market financial forecasting. Nevertheless, this system accepts only evenly spaced time series data and is unable to utilize non-evenly spaced time series data or qualitative data such as “recommended” or “not recommended”, limiting its applications. Accordingly, there is a need for modifications to current solutions so that the solutions can accept other forms of data, extending its applications outside of financial and business forecasting. Summary HK 30135840 A 2 This document describes, in one embodiment, a five-step process for generating forecasts and recommendations from diverse quantitative and qualitative datasets is disclosed. The process includes a pre-processing module, vector autoregression (VAR) analysis, deep neural network (DNN) predictions, genetic algorithm (GA) modules, and a post-processing module. The input data types may include evenly spaced time series, non-evenly spaced time series, and non-numerical data such as customer evaluations in the words “recommended”, “not recommended”, “liked”, and “disliked”. The disclosed process accepts data and converts it into evenly spaced time series, a format acceptable for VAR analysis. The processed data is then automatically input into the VAR analysis module to identify the significant correlated and leading indicators. DNNs are trained on the selected data to produce forecasts and predictions. The GA module evaluates and selects the most suitable and best-performing DNN models to cope with the time-dependent and dynamic nature of some datasets. The post- processing model converts the output data into actionable recommendations. Brief Description of the Drawings The following description sets forth additional aspects, features, and advantages of the present invention, to be understood with reference to the drawing figures and detailed description herein. FIG. 1 illustrates a diagram depicting the five modules of the present invention from input to output. FIG. 2 illustrates a flowchart depicting the procedure of the pre-processing module. FIG. 3 illustrates the procedure of the genetic algorithm employed in the present invention. FIG. 4 illustrates a diagram depicting the three steps in the post-processing module. FIG. 5 illustrates the procedure of the hybrid cybernetic system in detail. Detailed Description Hereinafter, characteristic aspects and example implementations of the present disclosure are disclosed for descriptive and illustrative purposes. In one embodiment, as illustrated in Figure 1, the system comprises a pre-processing module (101), vector autoregression analysis module (102), deep neural network prediction module (103), genetic algorithm module (104), and a post-processing module (105). HK 30135840 A 3 Actionable suggestions and recommendations based on generated forecasts and predictions in finance, business, inventory and demand forecasting, customer or employee satisfaction metrics, patient motoring in a healthcare setting, traffic forecasting, and adaptive learning are produced through this embodiment and its ability to handle diverse numerical and non-numerical datasets. While the existing hybrid system including of a vector autoregression analysis module, neural network prediction modules, and a genetic algorithm module (steps 102, 103, and 104) can accept only evenly spaced multivariate time series, the present disclosure contains a pre-processing module that can convert non-evenly spaced time series and qualitative data into evenly spaced time series suitable for vector autoregression analysis. As illustrated in Figure 2, non-numerical data collected at various evenly spaced or non-evenly spaced time intervals can also be accepted into the pre-processing module, including qualitative evaluations like “strongly recommended”, “somewhat recommended”, “neutral”, and “strongly disagree”, as well as ratings such as the five-star rating system (201). This qualitative data may be encoded into numbers corresponding to a specific time points to perform vector autoregression analysis. In one embodiment, the system collects the data in module (201). Then, depending on whether the data is numerical, or with a natural order, modules 202, 203, 204 are deployed to handle different types of input data. Module (205) is to linearly interpolate missing values to produce evenly spaced time series. In module (206), processed time series data will serve as input into the vector autoregression analysis module. If the qualitative evaluations have no immediately obvious natural order, such as text-based customer reviews (“great service” or “strongly recommended”), natural language processing (NLP) and sentiment analysis is employed to produce an ordering (202). NLP, a field of computer science using machine learning, allows computers to interpret and understand natural human language through combining computational linguistics, machine and deep learning, and statistical modeling. To apply NLP to extract meaning from words and sentences, the text is first preprocessed by changing the text into lowercase and removing punctuation, then broken down into tokens (individual words or short pieces) to facilitate sentiment analysis. The words are then lemmatized by reducing words into their lexical root (for example, “running” or “ran” are reduced to “run”), then converted into numerical vectors based on frequency of occurrence using the bag-of-words model (BoW), which is required for machine-learning (ML) based sentiment analysis. HK 30135840 A 4 BoW accepts an input of preprocessed text including of many smaller documents and identifies the unique words in the text. Then, each document is assigned a ( × 1) vector with numerical entries based on the number of occurrences of each unique word, without accounting for the order of the words in the document. BoW is implemented using NLTK (Natural Language Toolkit), a Python library for NLP. Semantic analysis is then employed to understand the meaning of individual words, encompassing techniques used to label and identify categories of text. In particular, sentiment analysis is applicable, which utilizes various NLP algorithms to determine the emotional meaning (positive, negative or neutral) behind a block of text. ML based sentiment analysis, a method of NLP, relies on a model that is fed a large dataset of labelled examples for training, such as “The food was amazing”: very positive. To include more nuance beyond simply “positive” and “negative”, five categories, “very negative”, “negative”, “neutral”, “positive”, and “very positive” are created. These examples are representative of the actual qualitative data that is inputted into the process, such as customer reviews and social media comments. To run ML, the examples may be processed as indicated above through tokenization and BoW before the Naive Bayes Algorithm, a simple and quick algorithm that works well with text, can be used to train the model. The Naive Bayes Algorithm utilizes Bayes’ Theorem to classify data based on probabilities, expressed mathematically as: = × 1 where is the probability of A happening given that B is true, is the probability of B happening if A is true, () is the prior probability of A happening (before considering B), and () being the overall probability of B happening. First, the prior probabilities are calculated from the training dataset, where P(Positive) = (number of positive reviews) / (total reviews), and vice versa for negative reviews. Then, likelihoods of individual words appearing in each positive or negative review is calculated and stored in a lookup table. For example, the algorithm determines how often the word “amazing” appears in very positive reviews, and how often the word “rude” appears in negative reviews. To classify new data, the model calculates the probability that the review is one of the five categories using Bayes’ Theorem, and the probability that is the highest is the label that is assigned. These five HK 30135840 A 5 categories form the basis for a ranked classification system similar to the five-star rating system, for which ordinal encoding can be applied to assign numerical values to the data. On a Likert scale or five-star rating system, where evaluations have a natural order (for example, 4 stars is a more positive rating than 2 stars, ordinal encoding is utilized (203), in which each option is assigned a relative numerical value based on its semantic meaning, where poorer evaluations like 1 star are assigned lower numeric values and better evaluations like 5 stars are assigned higher numeric values. For instance, in an evaluation system with five categories (such as the 5-star rating system), a numerical value of 1 is assigned to the timestamp associated with a one-star review, and a numerical value of 3 is assigned to the timestamp associated with a three-star review, preserving the relative order of the reviews. Ordinal encoding can be implemented in Python using the Python data analysis library pandas. Afterward, the resampling and interpolation steps indicated below are applied to this non-evenly spaced time series data to ensure all values are evenly spaced time series. To convert non-evenly spaced time series into evenly spaced multivariate time series, resampling and interpolation are utilized. Functions available in data analysis library provides the functionality to upsample non-evenly spaced time series (204) into a desired time frequency (hourly, daily, etc.) through the methods, for example pandas.resample(), based on the existing data points, aiming to reduce the error associated with interpolating too many data points. Missing data points are then linearly interpolated using data analysis library, for example pandas, to produce a set of data spaced at equal time intervals (205). With sufficient neighbouring data points and smooth, non-cyclical trends such as temperature, stock prices, demand, and ratings, linear interpolation is a simple and accurate means of interpolating data points. Linear interpolation to determine data point at time from neighbouring data points 0(0, 0) and 1 1, 1 is represented mathematically as: = 0 + − 0 ∙ 1 − 0 1 − 0 2 where , 0 , and 1 represent timestamps, i.e. the number of seconds or days since a fixed reference point, and , 0, and 1 represent numerical data points. The inclusion of a pre-processing module that can interpolate enhances its applications, allowing them to extend beyond business and financial forecasting, as many real-world datasets of time series are irregularly spaced or missing key time steps. The output of the pre-processing module is numerical evenly spaced time series (206). HK 30135840 A 6 Vector autoregression (VAR) is a widely used modelling method for business forecasting, with applications in forecasting macroeconomic time series. VAR identifies the linear relationships and interactions between multiple different input variables as they change over time, treating all variables symmetrically without considering independence versus dependence. In a VAR model, each variable corresponds to an equation modelling its evolution over time, creating a system of equations with variables containing multivariate time series (MTS) and with coefficients estimated by the ordinary least squares (OLS) method. A system of equations in a VAR model can be expressed mathematically as: = + + 3 where is a ( × 1) vector of variables, is a ( × ) polynomial matrix in the backshift operator with lag length , such that 1 + 22 +… + . is a × 1 vector of constant terms, and is a ( × 1) vector of white-noise error terms. The lag length of produces an equation of the model that has × + 1 coefficients, allowing statistically insignificant variables and lags to be excluded and enabling the identification of statistically significant interdependencies between the MTS. In the present disclosure, VAR analysis is implemented with RATS, a software package for time series analysis and econometrics. Prior literature shows that conventional statistical techniques for time series forecasting, such as VAR, can work complementarily with neural network prediction modules (including deep neural network modules), while traditional techniques alone have already reached their limitations in forecasting applications. Deep neural networks (DNN) hold the capability to implicitly detect complex nonlinear relationships between independent and dependent variables while requiring less formal statistical training. DNNs mimic the workings of the human brain and include many interconnected layers: the input layer, multiple hidden layers, and output layer. Mathematically, the DNN structure can be expressed as: = ( =1 2 ( =1 1 )) 4 where denotes the number of inputs, the number of hidden neurons (small processing units that make up the layers of the DNN), (1) the weights between the input and hidden layers, and (2) the weights between the hidden and output layers. The sum of the products of (2) and each input ( ) HK 30135840 A 7 produces the outputs of the hidden layer ℎ = ( =1 (2)) . The outputs of the hidden layers are then passed through the output layer, producing the output . Long Short-Term Memory (LSTM) is a commonly used DNN model that has been shown to outperform traditional forecasting techniques such as univariate autoregressive model and autoregressive integrated moving average (ARIMA) in time series forecasting. LSTM includes three gates: a forget gate, input gate, and output gate. The forget gate removes information that is no longer useful through a sigmoid function with an output of either 0 (the information is forgotten) or 1 (the information is retained). The input gate adds useful new information using the sigmoid function to filter values similar to the forget gate, then a vector of new content is produced using the tanh function that is stored. The output gate extracts the useful information from the memory cell to be output, which is used as a final prediction or to be used in another step. While linear forecasting methods like VAR allow for ease of explaining the results, nonlinear models like DNNs lack transparency and ease of interpretation: in a DNN model, it is difficult to determine which input variables contributed most significantly to the final output, and the results of DNN predictions are more difficult to interpret than the odds ratios from the coefficients of a conventional regression model. Genetic algorithms (GA) draw heavily from the concept of natural selection first proposed by Charles Darwin and are parallel, adaptive search algorithms. GAs begin with an initial population of varying potential solutions and, over time, select for the optimal performance of a DNN after several generations of selecting the fittest and best performing models to cope with the time-dependent nature of the datasets. GAs have been shown to improve the performance of DNNs in forecasting, indicating their compatibility. In the GA procedure shown in Figure 2, () represents the population of prediction models at generation number , while represents the set of offspring models at generation number . Step 301 is the initialization step for a predefined number of input models. Step 302 evaluates the performance and fitness of each model. Step 303 selects well-performing models from the set of parent models to pass on their weightings to the offspring models, while poorly performing models are rejected. Step 304 produces a population of offspring models () that inherits its weights from the best performing parent models that is then evaluated like in step 302. Step 305 creates a new population of models ( + 1) , combining the most optimal traits from both () and () . Step 306 repeats steps 303, 304, and 305 in the next generation + 1 until the termination condition is met. HK 30135840 A 8 Interpreting the forecasted results of the previous four modules and providing actionable suggestions is done by the post-processing module immediately following the GA module’s selection process. Once the best performing DNN model with the most optimal weightings is selected, that DNN module is used to forecast predictions from the output of the VAR analysis, yielding raw numerical outputs. Three post- processing steps are followed, as per Figure 4: 1. Trends and patterns in the numerical data are analyzed (Step 401) 2. Numerical outputs are decoded back into qualitative values, if applicable (Step 402). 3. Actionable recommendations and suggestions in natural text are produced by a trained natural language generation (NLG) model (Step 403). Trends and patterns in the DNN’s forecasted results, including rates of change, cyclical patterns, turning points, and noise are detected and analyzed in the post-processing module. First, the smoothing algorithm simple moving average (SMA) is applied to reduce short-term noise, making long-term trends clearer. The SMA of the last data points 0… at time is produced by the mean of the previous values (including the current value), represented mathematically as: = 1 =0 −1 −1 5 which, for a dataset containing points, yields a set of − + 1 SMAs that can then be graphed and analyzed. This algorithm is implemented using the data analysis library, for example Python library pandas. Then, a trend line is fitted to the graph of the SMAs, using either linear or polynomial regression based on the shape of the graph to determine the direction of the trend through the value of its slope. Trends over different time intervals, such as week-over-week or month-over-month are also analyzed using the above techniques. If the original data input was qualitative (e.g. ratings or customer evaluations) that was encoded into numerical values in the pre-processing module, a Python dictionary containing sets of numerical values as keys and qualitative labels as values can decode those rounded numbers into their original qualitative labels. A generative pretrained transformer (GPT), which is a learned NLG model, is utilized to interpret the trends and results and produce a summary in natural text, as well as actionable recommendations based on those results. For ease of implementation without large investments into training a model, HK 30135840 A 9 few-shot prompting is used by inputting a few examples of sample desired input and output, mostly relying on the GPT’s pre-trained knowledge to recognize the pattern. Figure 5 illustrates the entire process of an embodiment of the hybrid cybernetic system. The five modules of the system described in this document represent only the broad conceptual operation of such individual modules without specific changes or limitations. Example Environments Supporting Modules (102, 103, 104) An example environment can be found based on what was proposed by the two business case studies carried out. The first one is on the tourism demand forecasting for the Hong Kong market. The second business case study is the modeling and forecasting of the Asian Pacific stock markets. The way our system was used in this environment was through the multivariable time series data being investigated with the vector autoregression (VAR) analysis, and then the relevant variables determined during this analysis were fed to the neural network (NN). Lastly, the genetic algorithm (GA) is used to cope with the time-dependent nature of the co-relationships among the variables. Through our system, experimental results show that more robust and accurate prediction than the benchmark NN were produced. This hybrid system is designed to automate the process of selecting input variables, making the NN module more transparent and improving the prediction performance with dynamical cross-market lead-lag interactions. In the first business case, tourist demand forecasting, different models have been developed separately for making predictions. Results show that the hybrid system described in this document is robust to the changes of the cross-market dynamics. The mechanism of learning and modeling is achieved with tools from econometrics and machine learning. This embodiment of the cybernetic system can discover and model the various forms of interaction between systems, like the financial markets and the tourist markets in the two experiments. The multi-network system can provide multivariate time series data, international tourist arrival data, tourism expenditures, population and gross domestic product (GDP) statistics, and historical financial market data. In one application, tourist arrival data may include monthly or quarterly visitor numbers to a target destination—such as Hong Kong—from source countries like Japan, Southeast Asia, the United States, HK 30135840 A 10 and Canada. Accurate forecasting of tourism demand is crucial for planning in areas like hotel bookings, food and transportation services, and airline seat allocations. In contrast, traditional financial forecasting studies have typically focused on historical prices and trading volumes within a single market, using either linear statistical models or nonlinear machine learning techniques. The cybernetic system integrates the strengths of both approaches: the explanatory power and statistical rigor of econometrics, and the adaptability of NNs and GAs. This hybrid framework allows the system to respond effectively to the shifting dynamics of cross-market dependencies. Results show a 35% improvement in forecasting performance compared to a benchmark NN model. Financial data may include daily opening and closing stock prices from various markets, such as those in the United States (e.g. Nasdaq, S&P 500, DJIA) and the Asia-Pacific region (e.g. Hong Kong, Japan, Australia, Singapore). To analyze the interdependencies between these markets, financial experts commonly apply econometric tests. These include linear and nonlinear regression, generalized regression models, vector autoregression (VAR), panel data analysis, systems of simultaneous equations, and regressions with lagged variables. Econometric analysis typically begins with a set of theoretical propositions and aims to establish precise, deterministic relationships among economic variables. In addition to econometric models, nonlinear systems such as neural networks have become increasingly prominent in forecasting tasks due to their strong pattern recognition capabilities. In financial applications, neural networks have been employed to predict variables such as exchange rates, interest rates, futures prices, stock indices, and property values. Since then, growing research has explored the potential of neural networks for asset price prediction. Comparative studies have shown that neural networks often outperform, or at least match, the accuracy of traditional regression-based approaches. The way traditional systems work individually vary, but they each carry an important role. VAR performs statistical tests to identify lead-lag relationships and interdependencies among variables. For instance, the VAR analysis can determine whether visitor arrivals from one country are influenced by economic indicators or population changes from another country. Similarly, it may quantify dependencies among financial markets, identifying whether movements in the S&P index lead movements in the Hong Kong stock market. In an example use case, tourist arrivals to Hong Kong from the U.S. are forecasted using a hybrid VAR- NN-GA system. During periods when there is a strong correlation between U.S. population size and visitor numbers, models that include population data achieve higher predictive accuracy. Conversely, in HK 30135840 A 11 periods of weak correlation, simpler models that exclude population variables perform better. The GA module is employed to simulate an expert’s ability to select the most appropriate model configuration for each sub-period. In this study, our automated system is designed to address this issue. The VAR module ensures that the input variables fed into the neural network have significant influence on the target variables. The GA module further enhances the system by selecting the most suitable model among several candidates. Given the dynamic and time-varying nature of business markets, it is essential to apply the best-fitting model to each sub-period. This adaptability is achieved through evolutionary computation, including the genetic algorithm, which can respond to changing market dynamics. By combining these three components, the hybrid VAR-NN-GA system automates the selection of input variables, forecasting computations, and evaluation of prediction models. The goal is to build on previous studies by incorporating additional insights from the movements of other leading markets. In a financial forecasting use case, opening prices in the Hong Kong and Japan stock markets are forecasted. The NN model, when fed with both t-1 opening and closing prices, shows lower error rates compared to models using t-1 opening prices alone. When cross-market relationships are strong (e.g., during periods of high correlation with U.S. markets), including U.S. market data as inputs improves forecast accuracy. Conversely, during periods dominated by local factors (e.g., the Asian Financial Crisis), models relying solely on domestic market data perform better. In each example, the hybrid forecasting system comprising the VAR, NN, and GA achieves superior forecasting accuracy compared to stand-alone neural network models. The hybrid NN-GA model is better than the benchmark stand-alone NN models. The overall percentage error of the proposed model is 13.47%, while the prediction errors of the two benchmark NN models are 14.49% and 13.74% respectively. The overall average error of the proposed model in this case study is 1.54%, while the prediction errors of the two benchmark NN models are 3.11% and 2.37% respectively. The results show that the performance of the proposed system is 35% better than that of the benchmark neural network model, the one feeding with data from Hong Kong. The output interfaces may present the forecasting results to users through dashboards, APIs, or embedded applications. These interfaces can support strategic planning in sectors such as hospitality, transportation, government policy, investment decision-making, and marketing. The described environment provides a flexible and adaptive forecasting framework for domains characterized by nonlinear and time-dependent cross-variable dynamics, such as tourism demand and financial markets. HK 30135840 A 1 Claims What is claimed is: 1. A hybrid neural network system to produce forecasts and recommendations from diverse data sources comprising: a pre-processing module to convert non-evenly spaced time series and qualitative input data into evenly spaced time series through interpolation, resampling, and ordinal encoding; and a vector autoregression analysis module to identify statistically significant correlated and leading indicators between input variables; and a deep neural network prediction module using the long short-term memory model to generate predictions and forecasts; and a genetic algorithm module to adjust the weightings of each neural network model based on model performance; and a post-processing module using natural language processing to interpret the neural network prediction results to inform suggestions and recommendations. 2. The hybrid neural network system method according to claim 1, wherein a vector autoregression analysis module pre-selects input variables having statistically significant influence on the target variable prior to processing by the neural network module. 3. The hybrid neural network system method according to claim 1, wherein the deep neural network is a long short-term memory network including its characteristic three gates: forget gate, input gate, and output gate. 4. The hybrid neural network system method according to claim 1, wherein a genetic algorithm module selects the most appropriate model configuration for each sub-period with different correlation structures. HK 30135840 A 1 HK 30135840 A 2 HK 30135840 A 3 HK 30135840 A 4 HK 30135840 A 5 HK 30135840 A

Images