Generating synthetic data for machine learning training and momentum trading using deep reinforcement learning

Abstract

Embodiments include using actual data to generate synthetic data suitable for training machine learning models, including Deep Reinforcement Learning (DRL) models. Some embodiments include receiving an actual dataset with a time-ordered sequence of historical data, and based on the actual dataset, generating one or more synthetic datasets. Embodiments further include training machine learning models with data comprising the actual dataset and the synthetic datasets. After model training, some embodiments include using the machine learning model to forecast future data based on the historical data and the synthetic data. Some embodiments additionally include managing a portfolio of securities, including creating a candidate set of high momentum securities based on momentum scores and using two or more DRL models to generate trading signals for the candidate set. The trading signals are used to manage the portfolio by adding or removing securities to / from the portfolio and / or adjusting their weights in the portfolio.

Description

GENERATING SYNTHETIC DATA FOR MACHINE LEARNING TRAINING AND MOMENTUM TRADING USING DEEP REINFORCEMENT LEARNINGCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to: (i) U.S. App. 18 / 675,228, titled “Generating Synthetic Data For Machine Learning Training,” filed on May 28, 2024, and currently pending;(ii) U.S. App. 18 / 778,937, titled “Momentum Trading Using Deep Reinforcement Learning,” filed on Jun. 20, 2024, and currently pending; and (iii) Indian Provisional App. No.202311052251 titled “Momentum Trading Using Deep Reinforcement Learning Comprising Synthetic Peer Generation Techniques for Stock Price Forecasting,” filed on Aug. 3, 2023, and currently pending. The entire contents of U.S. App. 18 / 675,228, U.S. App. 18 / 778,937, and Indian Provisional App. No. 202311052251 are incorporated herein by reference.SUMMARY

[0002] The stock market can be very volatile. Prices of stocks tend to change asynchronously throughout the day. Investing in stocks both in the long term and short term can be very profitable if the investment timeline (buying and selling point) is properly chosen.However, investing in the stock market can be risky because of the underlying unpredictability of future prices. Nevertheless, price forecasting systems can provide insight into potential future prices so that investors can plan investments accordingly.

[0003] Automatic stock price forecasting systems can be very useful. Many different stocks can be traded in the public markets and their prices change second to second. One of the major challenges in training automatic forecasting systems is the unavailability of adequate training data both in terms of the volume of data available and the variety of data available. Disclosed embodiments present a technique sometimes referred to herein as EchoFlux that generates synthetic data for training the systems.

[0004] Experimental results described herein show that price prediction models that have been trained with synthetic data generated by the disclosed EchoFlux techniques perform better (i.e., make more accurate future predictions) than prediction models that have been trained with only actual historical data. Further, the experimental results show that training price prediction models with synthetic data generated by the disclosed EchoFlux techniques requires less training data than traditional approaches that use only actual historical data. Since models can be trainedwith less data, computing systems that implement price forecasting models training according to the disclosed EchoFlux techniques require less memory resources and require less training time (i.e., fewer processing cycles) as compared to computing systems that implement price forecasting models trained with traditional approaches. Thus, aspects of the disclosed EchoFlux techniques improve the technical operation of computing systems configured to implement price forecasting models as compared to traditional methods.

[0005] In operation, the EchoFlux techniques include generating synthetic peer stocks that move in tandem with the original stock (Echo) but are also different from the original stock in terms of variance (Flux). The synthetic data generated by the EchoFlux techniques provides a more robust dataset for training a Machine Learning-based stock price forecasting system by combining synthetic data for one or more synthetic peer stocks along with the original data for an actual stock. As mentioned earlier and described further herein, stock price forecasting models trained using the synthetic peers (and their corresponding synthetic data) have been found to be more accurate in terms of average deviation from actual prices than traditional approached that use only actual historical data for model training.

[0006] There have been several attempts at forecasting stock prices, most of them targeted towards the end of day closing price. Examples of earlier attempts include those described in: (i) Staffini, A., Stock price forecasting by a deep convolutional generative adversarial network, Frontiers in Artificial Intelligence, 5, 837596 (2022); (ii) Vijh, M., Chandola, D., Tikkiwal, V. A., & Kumar, A., Stock closing price prediction using machine learning techniques, Procedia computer science, 167, 599-606 (2020); (iii) Pratheeth, S., & Prasad, V, Stock Price Prediction using Machine Learning and Deep Learning, 2021 IEEE Mysore Sub Section International Conference (MysuruCon), pp. 660-664 (October 2021); (iv) Varaprasad, B. N., Kanth, C. K., Jeevan, G., & Chakravarti, Y. K., Stock Price Prediction using Machine Learning, 2022 International Conference on Electronics and Renewable Systems (ICEARS), pp. 1309-1313 (2022); and (v) Shen, J., & Shafiq, M. O., Short-term stock market price trend prediction using a comprehensive deep learning system, Journal of Big Data, 7(1), pp. 1-33 (2020).

[0007] But none of these prior approaches has had significant success. Forecasting closing prices of stocks on a daily level is itself a difficult task and the difficulty is further magnified when quarterly data is used. One of the major reasons for this difficulty is the unavailability of data especially for stocks which have gone public recently because stocks that have only recentlybecome publicly traded generally have less historical price data available for analysis as compared to other more established companies with many years of stock price history. There are many new stocks which demonstrate high volatility like older stocks but training automatic forecasting systems for them is difficult due to unavailability of data (short historical price data).

[0008] Newer data is also very important for understanding the price movement of stocks (irrespective of the time when the stock became publicly traded) as all stocks (both newly-listed stocks and well-established stocks) are influenced by recent factors and often indicate the nature of price movement of a stock. In the case of forecasting quarterly stock prices using quarterly information, the number of available data points for training the systems is greatly reduced thereby making training more difficult.

[0009] To tackle this paucity of historical price data, different data augmentation schemes have been proposed. Examples of data augmentation schemes include those disclosed in: (i) Teng, X., Wang, T., Zhang, X., Lan, L., & Luo, Z., Enhancing stock price trend prediction via a time-sensitive data augmentation method, Complexity, pp. 1-8 (2020); and (ii) Yadav Vanguri, N., Pazhaniraj an, S., & Anil Kumar, T., Extraction of Technical Indicators and Data Augmentation-Based Stock Market Prediction Using Deep LSTM Integrated Competitive Swarm Eeedback Algorithm, International Journal of Information Technology & Decision Making, pp. 1-27 (2023).

[0010] But these prior approaches do not consider the concept of integrating a stock’s original information along with peer stocks. There are various attributes like belongingness to the same industry or sector, having similar market capitalization, similar price movement, etc. that can be considered for associating a stock as peer of another stock.

[0011] However, associating stocks as peers based on price movement is very difficult as different stocks often demonstrate very little correlation in terms of price movement. Price based peering, if it can be done, can enhance the effectiveness of forecasting systems by introducing another dimension of variability. Price-based peering can also help supplement for the unavailability of data.

[0012] Creating a synthetic peer of a particular stock involves generating a synthetic market data series that closely mirrors the pattern of real price sequences. One classical approach to the problem is to create a Vector Autoregression Model as described in Liitkepohl, H., Vector autoregressive models, Handbook of research methods and applications in empiricalmacroeconomics, p. 30 (2013). In Liitkepohl, lagged values of the Open, High, Low and Close prices are used to predict the current values. Another approach favored by some researchers is to stitch together sub-samples of the real data series in a varying time-order as described in Kinlay, J., Synthetic Market Data and Its Applications (2023), available at SSRN 4380552. But while these approaches have been used in connection with forecasting stock prices, these techniques have not been used for generating synthetic stock price data that is used for training machine learning models to forecast stock prices in the manner described herein.

[0013] Embodiments disclosed herein include developing synthetic peers of a particular stock using the novel EchoFlux techniques. The EchoFlux techniques include statistical approaches that treat synthetic price data of a stock as an “echo” of its original counterpart having a “flux” component that makes the synthetic price data different from the original stock price data. One important aspect of the technique is generating synthetic peer stocks that move in tandem with the original stock thereby making available a bigger and more robust dataset for training machine learning models by combining multiple peers with the original data. The stock price forecasting models trained using the synthetic peer data generated according to the disclosed approaches have been found to be more accurate in terms of average deviation from actual price.

[0014] In some embodiments, the EchoFlux technique includes generating a pool of synthetic peer stocks Pi . . ..PN based on the data of an original stock S. The synthetic peers adhere to the nature of price movement of S and at the same time introduce variability. This allows for forecasting stock prices with lower error.

[0015] A stock’s data (e.g., daily prices or other data) can be represented as an ordered sequence of m points [si, S2, ...., sm], where every smrepresents the stock’s price on the mthday. This technique is able to generate multiple peer sequences each of which has m points. The proposed bi-stage transformation of Si to pi is presented as follows: pi= (si where: p; represents the ithsample of the peer stock P, generated from the ithsample of the original stock S;9i represents the maximum deviation percentage (magnitude) of pi from s;; and \| / i represents the associativity of 9i with Si.

[0016] In operation, 0 < 0i < L, where 0i G R. \| / i G [+, -], and where L denotes the maximum deviation. Thus if \| / i = +, then Vp,s : p > s. Similarly, if \| / i = -, then Vp,s : p < s. A higher value of L allows higher deviation of peers from the original data. Using the y values, peers \| / vqare generated where v G {{+}, {-}, {+,-}}, q G R and |q| represents the number of peer stocks.

[0017] In some embodiments, to determine an acceptably representative peer group for a stock ST, q peers V v are generated leading to 3*q peers. For example, 15 peers would be generated when q = 5 and where each set of peers includes (i) a first peer where the synthetic price for the first peer at point m is less than the actual price of the actual stock at sm(see, e.g., Figs. 1-2). (ii) a second peer where the synthetic price for the second peer at point m is greater than the actual price of the actual at sm(see, e.g., Figs. 3-4); and (iii) a third peer where the synthetic price for the third peer at point m is greater than or less than the actual price of the actual stock at sm(see, e.g., Figs. 5-6). Unordered combinations of the 3*q peers along with the original data are tested to determine an acceptably representative peer combination to enhance forecast performance. The associated figures along with the used methodology are described herein.

[0018] In some embodiments, the trading data generated by the approaches summarized above and described further herein can also be used to train algorithmic trading systems, including but not limited to Deep Reinforcement Learning (DRL) algorithms used with algorithmic trading systems, such as the systems and methods for momentum trading using DRL algorithms disclosed herein. However, the disclosed techniques for generating synthetic trading data can be used on their own separately from the disclosed momentum trading techniques. Similarly, the disclosed momentum trading techniques can be used separately from the disclosed techniques for generating synthetic trading data. Nevertheless, using the trading data generated according to the disclosed trading data generation techniques to train DRL algorithms for use in the disclosed momentum trading techniques can, in some instances, provide further improvements over prior approaches as compared to using the trading data generation techniques and momentum trading techniques independently.

[0019] As mentioned above, the present application also discloses and describes several new and efficient strategies for portfolio management within the domain of Quantitative Trading (QT) by combining a momentum trading screening approach with an innovative stock selection methodology based on Deep Reinforcement Learning (DRL).

[0020] Quantitative trading (QT) is an investment strategy that takes a systematic approach driven by mathematical models and data analysis. In contrast to the ad hoc nature of traditional discretionary trading, QT is a more scientific and objective method for decision-making in financial markets. By relying on algorithms and quantitative analysis, QT offers a structured framework for evaluating and executing trades, enhancing precision, and minimizing emotional biases. Aspects of QT are described in E. P. Chan, “Quantitative trading: how to build your own algorithmic trading business,” John Wiley & Sons (2021).

[0021] With recent advances in the capabilities of artificial intelligence (Al) and machine learning (ML) technologies, QT has gained popularity in recent years, commanding a substantial share of trading volumes in both developed and developing markets. For instance, in developed markets such as the U.S., QT accounts for over 70% of trading volumes, while in developing markets like China, it constitutes approximately 40% of the trading volumes, as explained in S. Sun, R. Wang, and B. An, “Reinforcement learning for quantitative trading,” ACM Transactions on Intelligent Systems and Technology, vol. 14, no. 3, pp. 1-29 (2023).

[0022] Traditional QT strategies, including momentum strategies, are popular methods employed by traders and investors. Aspects of traditional QT strategies, including momentum strategies, are discussed in C. R. Harvey, S. Rattray, and O. Van Hemert, “Strategic risk management: Designing portfolios and managing risk,” John Wiley & Sons (2021). However, momentum strategies tend to only perform well in specific market conditions as explained in Daniel, Kent, and Tobias J. Moskowitz, “Momentum crashes,” Journal of Financial Economics, Vol. 122, No. 2, pp. 221-47 (2016). Another approach to trading involves utilizing signals from machine learning-based system, as described in (i) Bicheng Wang & Xinyi Zhang, “Deep learning applying on stock trading,” CS230: Deep Learning, Stanford University, CA (Spring 2021); (ii) C. Tian, “Applications of Machine Learning and Reinforcement Learning in Investment and Trading,” University of California Irvine (2020); (iii) Hambly, Ben M. and Xu, Renyuan and Yang, Huining, “Recent Advances in Reinforcement Learning in Finance” (Feb. 27, 2023); (iv) Huang, Wei, Yoshiteru Nakamori, and Shou-Yang Wang, “Forecasting stock market movement direction with support vector machine,” Computers & Operations Research, Vol. 21, No. 10, pp. 2513-2522 (2005); and (v) Abe, Masaya, and Hideki Nakayama, “Deep learning for forecasting stock returns in the cross-section,” Pacific- Asia Conference on Knowledge Discovery and Data Mining, (Jan. 3, 2018).

[0023] For example, supervised learning algorithms like the tree-based models described in G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA (Dec. 4, 2017), the support vector machines (SVM) described in Huang, Wei, Yoshiteru Nakamori, and Shou-Yang Wang, “Forecasting stock market movement direction with support vector machine,” Computers & Operations Research, Vol. 21, No. 10, pp. 2513-2522 (2005), and the deep neural networks in Abe, Masaya, and Hideki Nakayama, “Deep learning for forecasting stock returns in the cross-section,” Pacific-Asia Conference on Knowledge Discovery and Data Mining, (Jan. 3, 2018) have been used for price forecasting. In addition, Natural Language Processing (NLP) techniques that enable sentiment analysis of news articles and social media posts can offer helpful insights for making trading decisions as described in Wenbin Zhang & Steven Skiena, “Trading Strategies to Exploit Blog and News Sentiment,” Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, Vol. 4. No. 1., pp. 375-378 (2010).

[0024] There is growing demand for data-driven machine-learning techniques across various domains. One subfield of data-driven machine-learning techniques is Reinforcement Learning (RL), which offers a mathematical framework for learning from experience by iteratively interacting with an environment, taking actions, receiving feedback, and adjusting strategies to maximize cumulative rewards. Deep Reinforcement Learning (DRL) is an extension of RL that incorporates deep learning techniques to handle complex and high-dimensional data.

[0025] The present application discloses and describes new and efficient strategies for portfolio management within the domain of QT by combining a momentum trading screening approach with an innovative stock selection methodology based on DRL.

[0026] Momentum strategies rely on the assumption that existing market trends will continue. More particularly, momentum strategies are based on the idea that assets that have performed well in the recent past will continue to perform well in the near future, while assets that have performed poorly in the recent past will continue to perform poorly in the near future. The underlying principle is grounded in the assumption that market trends persist due to various factors such as investor sentiment, institutional buying or selling pressure, herd mentality and fundamental factors (e.g., consistent earnings growth, market share expansion, etc.), all contributing to the asset’s performance.

[0027] However, momentum strategies come with inherent risks because stocks can abruptly reverse their direction, leading to significant losses if not managed properly. The DRL based techniques disclosed herein can help mitigate this risk by identifying early signs of potential reversals that traditional momentum strategies tend to miss. Even when a stock exhibits strong momentum (up or down), its price movement is not always smooth; the stock price can often experience pullbacks and consolidations within a long-term trend. But by accurately identifying these price movements, the DRL based approaches disclosed herein can provide more timely entry and exit signals, making the novel approaches disclosed herein more robust and adaptive to complex market dynamics as compared with the traditional approaches described in the literature identified above.

[0028] Accordingly, one aspect of the disclosed systems and methods includes, among other features, training a plurality (two or more) of Deep Reinforcement Learning (DRL) based machine learning models with a training dataset comprising one or more (or all) of (i) technical data relating to individual securities in a set of securities, (ii) fundamental data relating to the individual securities in the set of securities, (iii) macroeconomic data, and / or (iv) news relating to the individual securities in the set of securities and / or macroeconomic data from social media, newspapers, magazines, blog posts, or similar information sources.

[0029] In some embodiments, the set of securities includes a set of stocks, and in some instances, a large set of stocks. In some embodiments, the set of securities may additionally or alternatively include other securities such as bonds, mutual funds, index funds, options or other derivatives, and / or other assets. For example, the set of securities includes one or more or of (i) some or all of the stocks of the Dow Jones Industrial Average index, (ii) some or all of the stocks NASDAQ 100 index, (iii) some or all of the stocks of the S&P 500 index, (iv) any combination of the foregoing, and / or (v) any other set of stocks. Most of the examples and embodiments disclosed herein refer to stocks, but the disclosed embodiments are equally applicable to any other type of security now known or later developed that can be traded on an exchange.

[0030] Examples of DRL based machine learning models compatible for use with the disclosed systems and methods include (but are not limited to): (i) an Advantage Actor Critic (A2C) model, (ii) a Deep Q-Networks (DQN) model, and / or (iii) a Proximal Policy Optimization (PPO) model.

[0031] For example, some embodiments include, among other features, for a first Deep Reinforcement Learning (DRL) based machine learning model in a plurality of DRL based machine learning models, generating a first plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. And for a second DRL based machine learning model in the plurality of DRL based machine learning models, generating a second plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. The first plurality of trading strategies is different than the second plurality of trading strategies at least in part because the first and second pluralities of trading strategies are generated via different reinforcement learning algorithms (or at least differently-configured implementations of the same reinforcement learning algorithm) as described herein.

[0032] The first plurality of trading strategies and the second plurality of trading strategies associated with the first and second DRL based machine learning models, respectively, are themselves based on trading policies learned by the DRL based machine learning models. In operation, a DRL agent generates (i.e., learns) a plurality of DRL trading policies by using a DRL algorithm to interact with the aforementioned training dataset comprising (i) the technical features corresponding to individual securities in the set of securities, (ii) the fundamental features of individual securities in the set of securities, and (iii) the macroeconomic data. The trading strategies implemented by an individual DRL based machine learning model are based on the DRL trading policies learned by the DRL agent.

[0033] An individual DRL trading policy maps a state to an action. In the context of a DRL trading policy, a state corresponds to a combination of (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. In some embodiments, an action in the context of a DRL trading policy corresponds to (i) purchasing shares of a security, (ii) selling shares of a security, and / or (iii) a neutral action, such as holding shares of a security that was previously purchased, or not purchasing any new shares of a security. In some embodiments, an action in the context of a DRL trading policy corresponds to (i) opening or closing a long position in a particular security, (ii) opening or closing a short position in a particular security,and / or (iii) taking no position, e.g., holding cash. In some instances, holding cash amounts to not opening a long position in a security or not closing a short position in a security.

[0034] In some embodiments, an individual trading strategy is based on one or more DRL trading policies learned by the DRL agent. For example, in some instances, a trading strategy corresponds to the implementation of a single DRL trading policy. In other instances, a trading strategy corresponds to the implementation of a combination of two or more DRL trading policies. In still other instances, a trading strategy corresponds to the implementation of one or more DRL trading policies with one or more other rules or constraints, such as rules or constraints that limit the portfolio weight for a certain security.

[0035] Some embodiments additionally include determining a momentum score for each security in the set of securities. Determining the momentum scores is separate from training the DRL based machine learning models to learn trading strategies. In operation, the disclosed embodiments use the momentum scores for the securities to determine the securities in the set of securities to which the trading strategies will be applied. Based on the determined momentum scores, a candidate subset of securities is selected from the set of securities. The candidate subset of securities is sometimes referred to herein as simply the “candidate subset.” In some embodiments, the candidate subset includes securities having a momentum score that satisfies a particular momentum threshold.

[0036] For example, the candidate subset may include (i) securities having a momentum score above a first (or positive) momentum threshold as a candidate security for a long position (e.g., a candidate to buy), and (ii) securities having a momentum score below a second (or negative) momentum threshold as a candidate security for short position (e.g., a candidate to sell or a candidate for a short position). In some embodiments, the total set of candidate securities in the candidate subset may be limited to some maximum number of securities. For example, the candidate subset may include a maximum number of securities having (i) the highest momentum scores above the positive momentum threshold and / or (ii) the lowest momentum scores below the negative momentum threshold. As described further herein, some embodiments may consider percentile rankings as thresholds, by, for example, selecting stocks having the highest positive momentum scores (i.e., within some top percentile of positive scores) as candidates for long positions, and selecting stocks having the highest negative momentum scores (i.e., within some top percentile of negative scores) as candidates for short positions.

[0037] Next, the plurality of DRL based machine learning models is used to generate trading signals for each security in the candidate subset and then aggregating the trading signals from the plurality of DRL based machine learning models for each security in the candidate subset. For example, some embodiments include for each security in the candidate subset, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading strategies (based on a first set of DRL trading policies previously learned by a DRL agent via interaction with the aforementioned training dataset according to a first DRL algorithm) to data associated with the security, (ii) generating a second set of one or more trading signals for the security by applying the second plurality of trading strategies (based on a second set of DRL trading policies previously learned by a DRL agent via interaction with the aforementioned training dataset according to a second DRL algorithm) to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security. Some embodiments are described with reference to aggregating trading signals from two sets of trading signals for ease of illustration. In operation, embodiments may aggregate trading signals from more than two sets of trading signals generated by more than two DRL based machine learning models.

[0038] In practice, the DRL agents learn the DRL trading policies on which the trading strategies are based from a large set of historical data relating to all the securities in the set of securities, whereas the DRL based machine learning models apply the trading strategies to a comparatively smaller set of current data (e.g., real time or substantially real time data) relating to the securities in the candidate subset. In this manner, the DRL based machine learning models apply the trading strategies derived from trading policies the DRL agents learned from the large set of historical data for the large set of securities to a comparatively smaller set of current data for a smaller, targeted set of securities, i.e., a the candidate set containing securities with high momentum scores.

[0039] In some embodiments, the aggregated trading signal also indicates a signal strength. For example, if the first DRL based machine learning model generated a “buy” signal for a first security, and the second DRL based machine learning model also generated a “buy” signal for the first security, then the aggregated trading signal for that first security would be stronger than the aggregated trading signal for a second security where the first DRL based machine learningmodel generated a “buy” signal but the second DRL based machine learning model generated a “neutral” or “sell” trading signal.

[0040] After generating the aggregated trading signals for each security in the candidate subset, some embodiments additionally include one or more of: (i) opening a long position (i.e., buying one or more shares) of a security in the candidate subset (e.g., adding shares of the security to a trading portfolio by purchasing those shares from the market), (ii) closing a long position (i.e., selling one or more shares) of a security in the candidate subset (e.g., removing shares of the security from the trading portfolio by selling those shares to the market, to the extent that the trading portfolio included one or more shares of that security), (iii) opening a short position (i.e., selling short one or more shares) of a security in the candidate subset, and / or (iv) closing a short position (i.e., buying back one or more shorted shares) of a security in the candidate subset (to the extent that the trading portfolio included one or more shorted shares of that security).

[0041] Some embodiments disclosed herein provide improvements in the functioning of a computing system configured to implement QT strategies.

[0042] For example, prior QT strategies employing machine learning models such as the ones disclosed in the literature identified earlier herein tend to focus on a large set of securities, and include training and operating a separate machine learning model for each security on which the QT strategy is focused. As a result, prior QT strategies required developing, training, and operating a large number of machine learning models that are each optimized for a separate stock.

[0043] Rather than generating separate models for each stock, some disclosed embodiments include developing, training, and operating a comparatively smaller number of models that are applicable to a large set of stocks. This comparatively smaller number of models is then applied to data from a selected subset of high momentum stocks rather than a comparatively larger data set from a comparatively larger set of stocks. Thus, the combination of the model training strategy with the momentum stock screening strategy described herein results in fewer machine learning models analyzing a smaller set of stock data as compared to prior QT strategies. As a result, the computing systems configured to manage portfolios with the disclosed embodiments described herein require far less computing resources that computing systems configured to perform various portfolio management functions with prior QT strategies.

[0044] Certain examples described herein may include none, some, or all of the above described features and / or advantages. Further, additional features and / or advantages may be readily apparent to persons of ordinary skill in the art based on reading the figures, descriptions, and claims included herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0045] For a more complete understanding of the present disclosure and its features and advantages, reference is now made to the following description, taken in conjunction with the accompanying drawings.

[0046] Figure 1 shows an example method for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data is always less than the original stock prices in the original stock price data according to some embodiments.

[0047] Figure 2 shows a chart depicting prices of an original stock compared to synthetically generated prices of a synthetic peer created by the method shown in Figure 1 according to some embodiments.

[0048] Figure 3 shows an example method for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data is always greater than the original stock prices in the original stock price data according to some embodiments.

[0049] Figure 4 shows a chart depicting prices of an original stock compared to synthetically generated prices of a synthetic peer created by the method shown in Figure 3 according to some embodiments.

[0050] Figure 5 shows an example method for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are both (i) less than the original stock prices in the original stock price data and (ii) greater than the original stock prices in the original stock price data, according to some embodiments.

[0051] Figure 6 shows a chart depicting prices of an original stock compared to synthetically generated prices of a synthetic peer created by the method shown in Figure 5 according to some embodiments.

[0052] Figure 7 shows an example method for merging original stock data with synthetically generated stock data according to some embodiments.

[0053] Figure 8 shows a chart depicting prices of an original stock compared to synthetically generated prices of a synthetic peer created by the method shown in Figure 5 showing positive and negative deviations in the synthetic peer relative to the original stock according to some embodiments.

[0054] Figure 9A shows an example method for generating synthetic data for machine learning according to some embodiments.

[0055] Figure 9B shows additional aspects of the method of Figure 9A for generating synthetic data for machine learning according to some embodiments.

[0056] Figure 10 shows an overview of a portfolio management technique that implements aspects of a method of momentum trading using a plurality of deep reinforcement learning models according to some embodiments.

[0057] Figure 11 shows aspects of training an individual deep reinforcement learning model according to some embodiments.

[0058] Figure 12 shows a flow chart method for momentum trading using a plurality of deep reinforcement learning models according to some embodiments.

[0059] Figure 13 shows an example computing system configured for implementing aspects of the methods and processes disclosed herein.DETAILED DESCRIPTION OF THE FIGURES

[0060] Disclosed embodiments include, among other features, generating synthetic peer stock price data with respect to input stock price data. Some disclosed embodiments, and aspects thereof, are sometimes referred to as the “EchoFlux.” In operation, the disclosed techniques include generating synthetic peer stocks (with corresponding peer stock data) that moves in tandem with the original stock (Echo), but the synthetic peer stocks (and their corresponding data) are different from the original stock by a variance (Flux). Compared with traditional approaches, generating peer stock data results in a better dataset for training a Machine Learning-based stock price forecasting system by combining multiple peers along with the original data.

[0061] As mentioned above, the disclosed embodiments also include systems and methods for momentum trading using DRL algorithms disclosed herein. The disclosed techniques for generating synthetic trading data can be used on their own separately from the disclosed momentum trading techniques. Similarly, the disclosed momentum trading techniques can be used separately from the disclosed techniques for generating synthetic trading data. Nevertheless, using the trading data generated according to the disclosed trading data generation techniques to train DRL algorithms for use in the disclosed momentum trading techniques can, in some instances, provide further improvements over prior approaches as compared to using the trading data generation techniques and momentum trading techniques independently.

[0062] Aspects of generating synthetic trading data (sometime referred to herein as EchoFlux) are described first, followed by aspects of momentum trading using DRL algorithms.

[0063] Disclosed embodiments of the EchoFlux process can generate 3 types of synthetic stock data based on original stock data, including: (1) generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are always lower than the original stock prices in the original stock price data; (2) generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are always greater than the original stock prices in the original stock price data; and (3) generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are both (a) greater than the original stock prices in the original stock price data and (b) less than the original stock prices in the original stock price data.

[0064] When generating peer stock data that is lower in value than the original (or reference) stock, the EchoFlux process includes (i) traversing through multiple price points (and in some instances, every price point) of series (C) of the original stock and (ii) duplicating those price points from series (C) as series (S), but with variations. To create the variations in series (S) from series (C), for each of the price points in series (C), a random number (R) within a range (X, Y) is generated. Thereafter, the considered point in series (C) is decreased by R% of its value. This procedure is described with reference to Figure 1. An example of a synthetically generated series (S) along with its original series (C) is shown in Figure 2, where the solid line represents theoriginal series (C) and the dashed line represents the synthetically generated series (S). The X- axis represents time while Y-axis represents price.

[0065] When generating peer stock data that is lower in value than the original (or reference) stock, the EchoFlux process includes (i) traversing through multiple price points (and in some instances, every price point) of series (C) of the original stock and (ii) duplicating those price points from series (C) as series (S), but with variations. To create the variations in series (S) from series (C) in this scenario, for each of the points in series (C), a random number (R) within a range (X, Y) is generated. Thereafter, the considered point in series (C) is increased by R% of its value. This procedure is described with reference to Figure 3. An example of a synthetically generated series (S) along with its original series (C) is shown in Figure 4. Here, the solid line represents the original series (C) while the dashed line represents the synthetically generated series (S). The X-axis represents time while Y-axis represents price.

[0066] When generating peer stock data that intersects the value of the original (or reference) stock, the EchoFlux process includes (i) traversing through multiple price points (and in some instances, every price point) of series (C) of the original stock and (ii) duplicating those price points from series (C) as series (S), but with variations. To create the variations in series (S) from series (C), for each of the points in series (C), a random number (R) within range (X, Y) is generated. Additionally, another random number (P) is generated in the range (U,V). Thereafter, the considered point in series (C) is increased by R% of its value if P is even. Otherwise, the considered point in series (C) is decreased by R% of its value. This procedure is described with reference to Figure 5. An example of a synthetically generated series (S) along with its original series (C) is shown in Figure 6. Here, the solid line represents the original sequence while the dashed line represents the synthetically generated sequence. The X-axis represents time while Y- axis represents price.

[0067] Figure 8 shows synthetically generated stock prices indicated by a dashed line along with the original stock prices indicated by a solid line (as shown in Figure 6). Here, the X-axis represents time while Y-axis represents price. In Figure 8, the synthetically generated stock prices exceed the original stock prices at some points while the synthetically generated stock prices are lower than the original stock prices at other points. The points where the synthetic stock price is higher than the original stock price are represented using upward arrows while the contrary is represented using downward arrows. In operation, generating the synthetic stock dataaccording to the disclosed processes ensures (or at least increases the likelihood) that the standard deviation of the synthetically generated stock is different from that of the original stock.

[0068] The disclosed EchoFlux processes can generate any number of synthetic stocks for an original stock by varying the range of deviation and the type of synthetic peer (e.g., generating synthetic peer data that is lower than the original data, generating synthetic peer that is greater than original data; and / or generating synthetic peer data that intersects the original data). Different types of synthetic peers with different ranges can also be combined with the original stock price data for training Machine Learning-based models. The synthetic peer stocks can also be generated based on opening price, closing price, midday price, hourly price, and / or any other price at any other time.

[0069] The example embodiments now will be described more fully hereinafter with reference to the accompanying figures, in which certain example embodiments are shown. The components shown and described with reference to the figures may, however, be embodied in many different forms and should not be construed as limited to the embodiments illustrated herein. Rather, the example embodiments disclosed herein are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the inventions contained herein to those skilled in the art.

[0070] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0071] Although the terms first, second, third etc. may be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section.

[0072] As used herein, the singular forms “a,” “an,” and “the,” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” “includes” and / or “including,” and “have” and / or “having,” when used in this specification, specify the presence of stated features, regions, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, regions, integers, steps, operations, elements, components, and / or groups thereof.

[0073] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.1. Generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are always less than the original stock prices in the original stock price data

[0074] Figure 1 shows an example method 100 for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data is always less than the original stock prices in the original stock price data according to some embodiments.

[0075] Method 100 begins at block 102.

[0076] Next, at block 104, method 100 includes receiving a first series C of closing stock prices for an original (or reference) stock. Although the example in Figure 1 is based on closing stock prices, method 100 is equally applicable to other price data, including but not limited to opening daily stock prices, hourly stock prices, weekly stock prices, monthly stock prices, quarterly stock prices, any combination of the foregoing, or any other pricing data suitable for use in training a machine learning system according the embodiments described herein.

[0077] In operation, the first series C for the original stock is a time-ordered sequence of actual historical price data of an actual security during a timeframe. The security can be any security listed on a financial exchange, including but not limited to a stock, a bond, a derivative, a market index, or any other type of security now known or later developed that has historical pricing data. The first series includes a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe.

[0078] In some embodiments, block 104 also includes setting a lower flux limit X and an upper flux limit Y. In other embodiments, the lower flux limit X and the upper flux limit Y are set before method 100 starts. These upper and lower limits are used in connection with creating synthetic pricing data during loop 112 of method 100 described further below.

[0079] In some embodiments, setting the lower flux limit X and the upper flux limit Y at block 104 includes setting the lower flux limit X and the upper flux limit Y based on one or more user inputs. In some embodiments, the lower flux limit X and the upper flux limit Y are based on one or more features of the first series C, e.g., an average inter-day or intra-day price fluctuation, a median inter-day or intra-day price fluctuation, or other feature suitable for setting the lower flux limit X and the upper flux limit Y.

[0080] Block 106 includes initializing a Count variable equal to zero. This Count variable corresponds to an individual data point (e.g., a price) in the first series C of closing stock prices.

[0081] Block 108 includes setting a series length L equal to the length of the first series C of closing stock prices for the original stock. In this context, the series length L corresponds to the number of data points in the first series C. For example, if the first series C of closing stock prices for the original stock received at block 104 includes 500 data points (i.e., 500 days of closing prices), then setting the series length L at block 108 entails setting series length L to 500.

[0082] Block 110 includes creating a second series S of closing stock prices for a synthetic stock. This second series S is initially equal to the first series C of closing stock prices for the original stock. However, the individual datapoints in the second series S will be modified in loop 112. This synthetic stock is a fictional security that is not listed on any financial exchange.

[0083] After setting the second series S equal to the first series C, method 100 begins loop 112 to update the individual closing stock prices in the second series S of closing stock prices for the synthetic stock with new values. The new values generated during the execution of loop 112 will be included in a dataset for training the machine learning model.

[0084] Loop 112 of method 100 begins at block 114, which includes setting variable R to a random number between the lower flux limit X and the upper flux limit Y from block 104.

[0085] Next, loop 112 advances to block 116, where an updated value of S[Count] is set to the original value of S[Count] (which is the same value as C[Count] for the original stock) multiplied by (R / 100), which is then subtracted from the original value of S[Count],

[0086] For example, if the original value of S [0] is a value of 55, and the random number R is a value of 7, then block 116 includes updating the value of S [0] to a value of (55 - 55*(7 / 100)) = (55 - 3.85) = 51.15. Similarly, if the original value of S[0] is 126, and the random number R is a value of 5, then block 116 includes updating the value of S[0] to a value of (126 - 126*(5 / 100)) = (126 - 6.3) = 119.7. Because the original value of S[Count] is multiplied by (R / 100) and thensubtracted from the original value of S[Count], the updated value for S[Count] determined at block 116 will always be less than the original value for SfCount],

[0087] Next, loop 112 of method 100 advances to block 118, which includes updating the value for Count by one, i.e., setting Count = Count + 1.

[0088] Next, loop 112 advances to block 120, which includes comparing the value for Count from block 118 to the series length L from block 108. If the value for Count is less than the series length L, then loop 112 returns to block 114 to generate the next value for the second series S. But if the value for Count is not less than the series length L, then loop 112 ends and method 100 advances to block 122.

[0089] At block 122, method 100 returns an updated series S for the synthetic stock. This series of stock price data is a synthetic time-ordered sequence of historical price data for the fictional security. This time-ordered sequence S of synthetic historical price data includes a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe.

[0090] Next, method 100 ends at block 124.

[0091] Figure 2 shows a chart 200 depicting prices of an original stock compared to synthetically generated prices of a synthetic peer created by the method 100 shown in Figure 1 according to some embodiments.

[0092] Solid line 202 indicates the actual price history of the original stock as a function of time. Dotted line 204 indicates the synthetic price history of the synthetic stock as a function of time. At each of the datapoints 206a-j in the series, the synthetic price of the synthetic stock is lower than the actual price of the original stock.2. Generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are always greater than the original stock prices in the original stock price data

[0093] Figure 3 shows an example method 300 for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data is always greater than the original stock prices in the original stock price data according to some embodiments.

[0094] Method 300 begins at block 302.

[0095] Next, at block 304, method 300 includes receiving a first series C of closing stock prices for an original (or reference) stock. Although the example in Figure 3 is based on closing stock prices, method 300 is equally applicable to other price data, including but not limited to opening daily stock prices, hourly stock prices, weekly stock prices, monthly stock prices, quarterly stock prices, any combination of the foregoing, or any other pricing data suitable for use in training a machine learning system according the embodiments described herein.

[0096] In operation, the first series C for the original stock is a time-ordered sequence of actual historical price data of an actual security during a timeframe. The security can be any security listed on a financial exchange, including but not limited to a stock, a bond, a derivative, a market index, or any other type of security now known or later developed that is has historical pricing data. The first series includes a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe.

[0097] In some embodiments, block 304 also includes setting a lower flux limit X and an upper flux limit Y. In other embodiments, the lower flux limit X and the upper limit Y are set before method 300 starts. These upper and lower limits are used in connection with creating synthetic pricing data during loop 312 described further below.

[0098] In some embodiments, setting the lower flux limit X and the upper flux limit Y at block 304 includes setting the lower flux limit X and the upper flux limit Y based on one or more user inputs. In some embodiments, the lower flux limit X and the upper flux limit Y are based on one or more features of the first series C, e g., an average inter-day or intra-day price fluctuation, a median inter-day or intra-day price fluctuation, or other feature suitable for setting the lower flux limit X and the upper flux limit Y.

[0099] Block 306 includes initializing a Count variable equal to zero. This Count variable corresponds to an individual data point (e.g., a price) in the series of closing stock prices.

[0100] Block 308 includes setting a series length L equal to the length of the first series C of closing stock prices for the original stock. In this context, the series length L corresponds to the number of data points in the first series C. For example, if the first series C of closing stock prices for the original stock received at block 304 includes 500 data points (i.e., 500 days of closing prices), then setting the series length L at block 308 entails setting series length L to 500.

[0101] Block 310 includes creating a second series S of closing stock prices for a synthetic stock. This second series S is initially equal to the first series C of closing stock prices for theoriginal stock. However, the individual datapoints in the second series S will be modified in loop 312. This synthetic stock is a fictional security that is not listed on any financial exchange.

[0102] After setting the second series S equal to the first series C, method 300 begins loop 312 to update the individual closing stock prices in the second series S of closing stock prices for the synthetic stock with new values. The new values generated during the execution of loop 112 will be included in the dataset for training the machine learning model.

[0103] Loop 312 of method 300 begins at block 314, which includes setting variable R to a random number between the lower flux limit X and the upper flux limit Y from block 304.

[0104] Next, loop 312 advances to block 316, where an updated value of S[Count] is set to the original value of S[Count] (which is the same value as C [Count] for the original stock) multiplied by (R / 100), which is then added to the original value of SfCount],

[0105] For example, if the original value of S [0] is a value of 55, and the random number R is a value of 7, then block 316 would update the value of S[0] to a value of (55 + 55*(7 / 100)) = (55 + 3.85) = 58.85. Similarly, if the original value of S [0] is 126, and the random number R is a value of 5, then block 316 would update the value of S[0] to a value of (126 + 126*(5 / 100)) = (126 + 6.3) = 132.3. Because the original value of S[Count] is multiplied by (R / 100) and then added to the original value of S[Count], the updated value for S[Count] determined at block 316 will always be greater than the original value for S [Count],

[0106] Next, loop 312 of method 300 advances to block 318, which includes updating the value for Count by one, i.e., setting Count = Count + 1.

[0107] Next, loop 312 advances to block 320, which includes comparing the value for Count from block 318 to the series length L from block 308. If the value for Count is less than the series length L, then loop 312 returns to block 314 to generate the next value for the second series S. But if the value for Count is not less than the series length L, then loop 312 ends and method 100 advances to block 322.

[0108] At block 322, method 300 returns an updated series S for the synthetic stock. This series of stock price data is a synthetic time-ordered sequence of historical price data for the fictional security. This time-ordered sequence S of synthetic historical price data includes a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe.

[0109] Next, method 300 ends at block 324.

[0110] Figure 4 shows a chart 400 depicting prices of an original stock compared to synthetically generated prices of a synthetic peer stock created by the method 300 shown in Figure 1 according to some embodiments.[0U1] Solid line 402 indicates the actual price history of the original stock as a function of time. Dotted line 404 indicates the synthetic price history of the synthetic stock as a function of time. At each of the datapoints 406a-j in the series, the synthetic price of the synthetic stock is greater than the actual price of the original stock.3. Generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are both (i) less than the original stock prices in the original stock price data and (ii) greater than the original stock prices in the original stock price data

[0112] Figure 5 shows an example method 500 for generating synthetic stock price data for a synthetic stock based on actual stock price data for an original stock where the synthetic stock prices in the synthetic stock price data are both (i) less than the original stock prices in the original stock price data and (ii) greater than the original stock prices in the original stock price data, according to some embodiments.

[0113] Method 500 begins at block 502.

[0114] Next, at block 504, method 500 includes receiving a first series C of closing stock prices for an original (or reference) stock. Although the example in Figure 5 is based on closing stock prices, method 500 is equally applicable to other price data, including but not limited to opening daily stock prices, hourly stock prices, weekly stock prices, monthly stock prices, quarterly stock prices, any combination of the foregoing, or any other pricing data suitable for use in training a machine learning system according the embodiments described herein.

[0115] In operation, the first series C for the original stock is a time-ordered sequence of actual historical price data of an actual security during a timeframe. The security can be any security listed on a financial exchange, including but not limited to a stock, a bond, a derivative, a market index, or any other type of security now known or later developed that is has historical pricing data. The first series includes a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe.

[0116] In some embodiments, block 504 also includes (i) setting a lower flux limit X and an upper flux limit Y, and (ii) setting a lower P limit U and an upper P limit V. In other embodiments, one or more (or all) of the lower flux limit X, the upper limit Y, the lower P limit U, and the upper P limit V are set before method 500 starts. These the values for X, Y, U, and V are used in connection with creating synthetic pricing data during the execution of loop 512 described further below.

[0117] In some embodiments, setting the lower flux limit X and the upper flux limit Y at block 504 includes setting the lower flux limit X and the upper flux limit Y based on one or more user inputs. In some embodiments, the lower flux limit X and the upper flux limit Y are based on one or more features of the first series C, e.g., an average inter-day or intra-day price fluctuation, a median inter-day or intra-day price fluctuation, or other feature suitable for setting the lower flux limit X and the upper flux limit Y.

[0118] Block 506 includes initializing a Count variable equal to zero. This Count variable corresponds to an individual data point (e.g., a price) in the series C of closing stock prices.

[0119] Block 508 includes setting a series length L equal to the length of the first series C of closing stock prices for the original stock. In this context, the series length L corresponds to the number of data points in the first series C. For example, if the first series C of closing stock prices for the original stock received at block 504 includes 1000 data points (i.e., 1000 days of closing prices), then setting the series length L at block 508 entails setting series length L to 1000.

[0120] Block 510 includes creating a second series S of closing stock prices for a synthetic stock. This second series S is initially equal to the first series C of closing stock prices for the original stock. However, the individual datapoints in the second series S will be modified in loop 512. This synthetic stock is a fictional security that is not listed on any financial exchange.

[0121] After setting the second series S equal to the first series C, method 500 begins loop 512 to update the individual closing stock prices in the second series S of closing stock prices for the synthetic stock with new values. The new values generated during the execution of loop 512 will be used for training the machine learning model.

[0122] Loop 512 of method 500 begins at block 514, which includes (i) setting variable R to a random number between the lower flux limit X and the upper flux limit Y from block 504, and(ii) setting variable P to a random number between the lower P limit U and the upper P limit V from block 504.

[0123] Next, loop 512 advances to block 516 which includes determining whether the value of P set at block 514 is an even number or an odd number. When P is an even number, then loop 512 advances to block 518. When P is an odd number, then loop 512 advances to block 520.

[0124] At block 518 (when P is even), an updated value of S[Count] is set to the original value of S[Count] (which is the same value as C[Count] for the original stock) multiplied by (R / 100), which is then added to the original value of SfCount],

[0125] For example, if the original value of S[0] is a value of 55, and the random number R (set at block 514) is a value of 7, then block 518 includes updating the value of S [0] to a value of (55 + 55*(7 / 100)) = (55 + 3.85) = 58.85. Similarly, if the original value of S[0] is 126, and the random number R is a value of 5, then block 518 includes updating the value of S [0] to a value of (126 + 126*(5 / 100)) = (126 + 6.3) = 132.3. Because the original value of S[Count] is multiplied by (R / 100) and then added to the original value of SfCount], the updated value for S[Count] determined at block 518 will always be greater than the original value for SfCount],

[0126] At block 520 (when P is odd), an updated value of S[Count] is set to the original value of SfCount] (which is the same value as CfCount] for the original stock) multiplied by (R / 100), which is then subtracted from the original value of SfCount],

[0127] For example, if the original value of S[0] is a value of 55, and the random number R (set at block 514) is a value of 7, then block 520 would update the value of S[0] to a value of (55 - 55*(7 / 100)) = (55 - 3.85) = 51.15. Similarly, if the original value of S[0] is 126, and the random number R is a value of 5, then block 520 would update the value of S[0] to a value of (126 - 126*(5 / 100)) = (126 - 6.3) = 119.7. Because the original value of SfCount] is multiplied by (R / 100) and then subtracted from the original value of S [Count], the updated value for SfCount] determined at block 520 will always be less than the original value for SfCount],

[0128] After executing one of block 518 or block 520, loop 512 advances to block 522, which includes updating the value for Count by one, i.e., setting Count = Count + 1.

[0129] Next, loop 512 advances to block 524, which includes comparing the value for Count from block 522 to the series length L from block 508. If the value for Count is less than the series length L, then loop 512 returns to block 514 to generate the next value for the secondseries S. But if the value for Count is not less than the series length L, then loop 512 ends and method 500 advances to block 526.

[0130] At block 526, method 500 returns an updated series S for the synthetic stock. This series of stock price data is a synthetic time-ordered sequence of historical price data for the fictional security. This time-ordered sequence S of synthetic historical price data includes a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe.

[0131] Next, method 500 ends at block 528.

[0132] Figure 6 shows a chart 600 depicting prices of an original stock compared to synthetically generated prices of a synthetic peer stock created by the method 500 shown in Figure 5 according to some embodiments.

[0133] Solid line 602 indicates the actual price history of the original stock as a function of time. Dotted line 604 indicates the synthetic price history of the synthetic stock as a function of time. Because method 500 generates synthetic stock prices that are both higher and lower than the original stock prices, the synthetic stock prices at different datapoints within the series may be higher or lower than the original stock prices. For example, the synthetic price of the synthetic stock is greater than the actual price of the original stock at datapoints 606b, 606d, 606e, 606g, and 606i. And the synthetic price of the synthetic stock is lower than the actual price of the original stock at datapoints 606a, 606c, 606f, 606h, and 606j .4. Merging synthetic stock price data with original stock price data in a matrix suitable for use with training a machine learning model

[0134] Figure 7 shows an example method 700 for merging original stock data with synthetically generated stock data according to some embodiments.

[0135] Method 700 begins at block 702.

[0136] Next, at block 704, method 700 includes receiving a first series C of closing stock prices for an original stock. Although the example in Figure 7 is based on closing stock prices, method 700 is equally applicable to other price data, including but not limited to opening daily stock prices, hourly stock prices, weekly stock prices, monthly stock prices, quarterly stock prices, any combination of the foregoing, or any other pricing data suitable for use in training a machine learning system according the embodiments described herein.

[0137] In operation, the first series C for the original stock is a time-ordered sequence of actual historical price data of an actual security during a timeframe. The security can be any security listed on a financial exchange, including but not limited to a stock, a bond, a derivative, a market index, or any other type of security now known or later developed that is has historical pricing data. The series of closing stock prices includes a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe.

[0138] In some embodiments, block 704 also includes setting a value N corresponding to a number of generated peers. This number N corresponds to the number (i.e., quantity) of synthetic stocks (and their corresponding series of synthetic stock data generated by any of methods 100, 300, and / or 500) to be combined into a training dataset with the first series C of closing stock prices for the original stock. For example, if 10 synthetic peer stocks (having 10 corresponding sets of synthetic stock data) have been generated and are to be merged into a dataset with the first series C of data, then N is set to 10. Similarly, if 50 synthetic peer stocks (having 50 corresponding sets of synthetic stock data) have been generated and are to be merged into a dataset with the first series C of data, then N is set to 50.

[0139] Block 706 includes initializing a Count value equal to one, which corresponds to the first synthetic stock (i.e., synthetic peer) and its series of synthetic stock data to be merged (or combined) into the training dataset with the first series C of actual stock data for the original stock. After executing method 700, the training dataset will include (i) the first time-ordered sequence of stock price data for the original stock (i.e., the first series C) and (ii) N additional sets of time-ordered sequences of stock price data, i.e. one set of synthetic data for each of the N synthetic peer stocks having synthetic pricing data generated by any one or more of methods 100, 300, and / or 500.

[0140] Block 708 includes setting a series length L equal to the length of the first series C of closing stock prices for the original stock. In this context, the series length L corresponds to the number of data points in first series C of stock price data for the original stock, i.e., the number of data points in the first time-ordered sequence C of stock price data for the original stock. For example, if the first series C of closing stock prices for the original stock received at block 704 includes 500 data points (i.e., 500 days of closing prices), then setting the series length L at block 708 entails setting series length L to 500.

[0141] Next, method 700 advances to loop 712.

[0142] Loop 712 of method 700 begins at block 714, which includes setting a Countl variable to zero. The Countl variable in block 714 corresponds to an individual data point (e.g., a synthetic closing price) in an individual series of time-ordered price data for an individual synthetic peer. These individual synthetic data points (e.g., the generated / synthetic time-ordered closing prices) for the synthetic peer will be combined into a training dataset with the first series C of actual data points (e.g., the actual time-ordered closing prices) for the original / actual stock. This training dataset will be used to train a machine learning model (e.g., a regression model) configured to predict future prices for the actual stock.

[0143] Next, loop 712 advances to block 716, which includes receiving a series P of closing stock prices for a particular synthetic peer. In operation, the series P is a series of stock price data for an individual synthetic peer that has been generated by any of methods 100, 300, and / or 500. The individual datapoints in this series P will be added to a matrix that contains the individual datapoints in the first series C of the original stock on which the data in the series P is based.

[0144] Next loop 712 advances to subloop 718. Subloop 718 is a nested loop within loop 712 of method 700. In operation, subloop 718 adds each individual synthetic peer datapoint (i.e., each closing price) for the series P of synthetic data points to the matrix that includes the first series C of data points for the original stock.

[0145] At block 720, subloop 718 includes, adding a datapoint P[Countl] from peer data set P to the matrix that includes the first set C of data for the original stock.

[0100] After adding the datapoint P[Countl] from the peer dataset P to the matrix, subloop 718 advances to block 722, which includes incrementing the Countl value by one, i.e., setting Countl = Countl + 1.

[0146] Next, subloop 718 advances to block 724, which includes comparing the value for Countl from block 722 to the series length L from block 708. If the value for Countl is less than the series length L, then subloop 718 returns to block 720 to add the next datapoint P[Countl] from the peer dataset P to the matrix.

[0147] But if the value for Countl is not less than the series length L, there are no more datapoints in P to add to the matrix. So, subloop 718 ends at block 724 and loop 712 advances to block 726, which includes incrementing the Count value by one, i.e., setting Count = Count + 1.In operation, updating Count by one amounts to moving on to the next peer data set of the N total peer datasets.

[0148] Next, loop 712 advances to block 728, which includes comparing the value for Count from block 706 to N (i.e., the total number (quantity) of sets of peer data) from block 704. If the current value for Count (set at block 726) is less than the value of N (set at block 704), then loop 712 returns to block 714 to add the next series P of data points to the matrix. But if the current value for Count (set at block 726) is not less than the value of N (set at block 704), then loop 712 ends and method 700 advances to block 730, which includes returning the matrix of pricing data. The matrix returned at block 732 includes (i) the first time-ordered sequence of stock price data for the original stock (i.e., the first series C) and (ii) N additional sets of time-ordered sequences of stock price data, i.e., one set of synthetic data for each of the synthetic peer stocks generated by any one or more of methods 100, 300, and / or 500.

[0149] Next, method 700 ends at block 732.

[0150] Figure 8 shows a chart depicting prices of an original stock compared to synthetically generated prices of a synthetic peer stock created by the methods 100, 300, and / or 500 showing positive and negative deviations in the synthetic peer stock relative to the original stock according to some embodiments.

[0151] Solid line 802 indicates the actual price history of the original stock as a function of time. Dotted line 804 indicates the synthetic price history of the synthetic stock as a function of time. Because methods 100, 300, and 500 generate synthetic stock prices that are both higher and lower than the original stock prices, the synthetic stock prices at different datapoints within the series may be higher or lower than the original stock prices.

[0152] For example, the synthetic price of the synthetic stock (dotted line 804) is greater than the actual price of the original stock (solid line 802) at datapoints 806b, 806d, 806e, 806g, and 806i. And the synthetic price of the synthetic stock (dotted line 804) is lower than the actual price of the original stock (solid line 802) at datapoints 806a, 806c, 806f, 806h, and 806j .Arrows pointing up at a datapoint indicate where the deviation between the synthetic price relative to the original price is positive, and arrows pointing down at a datapoint indicate where the deviation between the synthetic price relative to the original price is negative.5. Example Methods for Generating Synthetic Stock Data

[0153] Figure 9A shows aspects of an example method 900 for generating synthetic data for machine learning according to some embodiments. Figure 9B shows additional aspects of the method 900 for generating synthetic data for machine learning according to some embodiments.

[0154] Method 900 begins at step 902, which includes receiving an actual dataset comprising a time-ordered sequence of actual historical price data of an actual security during a timeframe, where the actual security is listed on a financial exchange, and where the time-ordered sequence of actual historical price data comprises a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe. The actual security may be any one of (i) a stock, (ii) a bond, (iii) a derivative, (iv) a market index, or (v) any other security now known or later developed that is traded on a financial exchange and has pricing history.

[0155] Next, method 900 advances to step 904, which includes, based on the actual dataset, generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe, wherein each fictional security of the plurality of fictional securities is not listed on any financial exchange, wherein an individual synthetic dataset for an individual fictional security comprises a synthetic time-ordered sequence of historical price data for the fictional security, and wherein the time-ordered sequence of synthetic historical price data comprises a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe.

[0156] In some embodiments, generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe at step 904 includes, for an individual fictional security: (i) for each market price of the actual security at each time point of the plurality of time points during the timeframe, generating a synthetic price at the time point by multiplying the market price by a random number, wherein market prices at different time points are multiplied by different random numbers to generate the synthetic prices at the different time points; and (ii) ordering the generated synthetic prices based on their corresponding time points into a synthetic dataset for the individual fictional security. In some embodiments, the random number is within a range defined by a lower bound and an upper bound.

[0157] In some embodiments, generating a synthetic price at the time point by multiplying the market price by a random number comprises any one or more of: (i) multiplying the marketprice by a random number that is less than 1, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point; (ii) multiplying the market price by a random number that is greater than 1, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point; and / or (iii) generating a synthetic price at the time point by multiplying the market price by a first random number based on a second random number that is different than the first random number, where (a) when the second random number has a first attribute, the first random number is less than one, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point, and (b) when the second random number has a second attribute, the first random number is greater than one, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point. In some embodiments, the first attribute includes having an even value and the second attribute includes having an odd value. In some embodiments, the first attribute includes having a positive value and the second attribute includes having a negative value.

[0158] In some embodiments, generating the plurality of synthetic datasets for the plurality of fictional securities during the timeframe at step 904 includes, for an individual synthetic dataset for an individual fictional security: (i) applying a bi-stage transformation to each data point si of the actual dataset to produce a corresponding synthetic data point pi for the synthetic dataset, wherein the bi-stage transformation is defined by an equation p; = (siOi) y / iSi, where 9i represents a maximum deviation percentage of pi from Si, and \| / i represents an associativity factor that determines a direction of deviation that is positive or negative. In some such embodiments, generating the plurality of synthetic datasets for the plurality of fictional securities during the timeframe at step 904 additionally includes: (i) selecting a value for 0i within a predetermined range of 0 to L, where L is a predefined maximum deviation limit, and Oi is a real number between 0 and L; and (ii) setting \| / i to a positive or negative value, wherein (i) when \ i is positive, synthetic price pi is greater than or equal to actual price Si, and (ii) when y / i is negative, synthetic price pi is less than or equal to actual price Si.

[0159] Next, method 900 advances to step 906, which includes training a machine learning model with data comprising the actual dataset and the plurality of synthetic datasets, wherein after training the machine learning model, the machine learning model is configured to forecast a future price data of the actual security. In some embodiments, the machine learning model comprises a regression model.

[0160] In some embodiments, training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets at step 906 includes: (i) for the actual security, extracting one or more features from price data in the actual dataset; (ii) for the plurality of fictional securities, for each fictional security, extracting the one or more features from price data in the synthetic dataset for the fictional security; and (iii) training the model to predict the future price of the actual security based at least in part on the one or more extracted features from the price data in the actual dataset and the one or more extracted features from the price data in each synthetic dataset. In some embodiments, the one or more extracted features include technical indicators associated with the actual security and the fictional securities.

[0161] In some embodiments, the one or more extracted features include one or more of: (i) a Relative Strength Index (RSI) that quantifies a speed and change of price movement of the price data, (ii) a Moving Average Convergence / Divergence (MACD) using two exponential moving averages of different timeframes to identify a strength of a directional move in the price data, (iii) an Average Direction Index (ADX) that measures a strength and momentum of change in the price data, (iv) a Stochastic Oscillator that measures a current price relative to a price range over a number of periods, which is useful as a momentum indicator to show how a security’s price has changed over a given period of time, (v) a Simple Moving Average (SMA) that measures an average price over a specific time period, or (vi) a Standard Deviation that measures volatility in the price data.

[0162] Some embodiments of method 900 additionally include step 908, which includes after training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets at step 906, predicting a future price for the actual security.

[0163] Some embodiments additionally include steps 910 through 918. Steps 910 through 918 relate to updating the training data based on new historical data associated with the actual security. In operation, steps 910 through 918 may be performed after collecting some threshold number of new datapoints for the actual security. For example, steps 910 through 918 may be performed after collecting any of (i) one or more days of new datapoints, (ii) one or more weeks of new datapoints, (iii) one or more months of new datapoints, and / or (iv) one or more quarters of new datapoints.

[0164] Step 910 includes, after training the machine learning model, receiving a subsequent actual dataset comprising a new time-ordered sequence of actual historical price data for theactual security during a subsequent timeframe, wherein the subsequent actual dataset includes market price data of the actual security on the financial exchange at each time point of a plurality of time points during the subsequent timeframe.

[0165] After step 910, method 900 advances to step 912, which includes based on the subsequent actual dataset, generating a subsequent synthetic dataset for each fictional security during the subsequent timeframe, wherein an individual subsequent synthetic dataset for an individual fictional security comprises a time-ordered sequence of synthetic price data for the fictional security comprising a synthetic price of the fictional security at each time point of the plurality of time points during the subsequent timeframe.

[0166] Next, method 900 advances to step 914, which includes generating an updated actual dataset by appending the subsequent actual dataset with at least a portion of the previously- received actual dataset. In some embodiments, generating the updated actual dataset additionally includes removing a quantity of old datapoints equal to the quantity of new datapoints. For example, if the subsequent timeframe is one month, updating the actual dataset may include (i) adding the new month of datapoints to the actual dataset and (ii) removing the oldest month of datapoints from the actual dataset.

[0167] Next, method 900 advances to step 916, which includes generating a plurality of updated synthetic datasets by generating an updated synthetic dataset for each fictional security, wherein generating an updated synthetic dataset for an individual fictional security comprises appending the subsequent synthetic dataset for the fictional security with at least a portion of the previously-generated synthetic dataset for the fictional security.

[0168] At step 918, method 900 includes re-training the machine learning model on data comprising the updated actual dataset and the plurality of updated synthetic datasets, wherein after re-training, the machine learning model is configured to forecast future price data of the actual security.6. Validation and Experimental Results for Synthetic Stock Data

[0169] A stock price forecasting experiment was conducted in three phases to assess the efficacy of the stock prices predicted by a machine learning model trained with the synthetic price data generated by the methods shown and described with reference to Figures 1-9. The dataset included the 500 stocks in the S&P 500 and 252 days of price data, which is the averagenumber of working days in a year. A prediction was made for each of the actual stocks in the dataset. The predictions were made using regression models implemented with XGBoost (Extreme Gradient Boosting). XGBoost is an open-source software library that provides a regularizing gradient boosting framework for use with implementing machine learning regression models that can be trained to make future predictions based in part on historical data. For the testing, regression models implemented with XGBoost were trained with three different training datasets referred to as Phase One, Phase Two and Phase Three (described below) and the accuracy of the predictions that were generated with each training dataset were compared.6.1 Phase One

[0170] In Phase One, a stock price forecasting model was trained separately for each of the 500 stocks in the dataset. For each stock in the dataset, the entire available history of price data for the stock for was used to train the model for that stock. Standard technical features were computed on this historical price data, and the target was set as the closing price for the stock (i.e., the closing price of the ithday of the stock was set as the target for the features of the ithday for the stock). The daily features for each date and the corresponding target for each date are illustrated in the following table.

[0171] The forecasting performance for a single stock for a single day was evaluated as the unsigned percentage deviation of the forecasted price with respect to the actual price (error). The average error for 252 days (i.e., the testing period) was computed for each of the stocks. The average of this error across the 500 stocks was computed to assess the performance for Phase One where the entire available history of price data for each stock was used for training the model for each stock.6.2 Phase Two

[0172] In Phase Two, only the most recent 600 days of history was used in the dataset used to train the model for each stock. The test period for Phase Two was the same 252 day period that was used in Phase One. The training scheme and the evaluation was exactly same as that of Phase One. Phase Two was implemented to assess how well the models for each stock would perform when each model was trained with a smaller amount of data (e g., only the most recent 600 days of data for the stock) compared to the performance of the models when each model was trained with entire available history of price data for the stock.6.3 Phase Three

[0173] The Phase Three used (i) the same data that was used in Phase Two and (ii) synthetic data that was generated based on the 600 data points for each of the stocks. In operation, the synthetic data was generated according to the methods disclosed and described with reference to Figures 1-9. The length (i.e., number of datapoints) of this synthetic data was identical to the length (i.e., number of datapoints) of the actual data. The table below illustrates the relationships between the original data and the synthetic data, where OiSj refers to the ithdata point for the jthsynthetic data sequence.

[0174] In Phase Three, features were computed for each data sequence, and all the featuretarget pairs were joined to form the training set used for model training. The temporal resolution of the data from which the features were extracted did not affect the targets obtained from the data because the correlation between the extracted features and the corresponding targets took the form of (Features;, Target;).

[0175] The table below illustrates a representation of the final training set, where SiFjDk refers to the jthfeature of the kthinstance for the i111synthetic sequence.

[0176] Post evaluation of the three phases (trained using same features, same test set) revealed that the highest error (1.43%) was obtained in Phase One, i.e., when the entire available history of price data for the stock for was used to train the model for that stock.

[0177] The next-lowest error (1.42%) was obtained in Phase Two, i.e., when only the most recent 600 days of price data for the stock for was used to train the model for that stock. Using only the most recent 600 days of price data ensured that the model did not get confused with older data.

[0178] The best performance, i.e., having the lowest error (1.35%), was obtained in Phase Three, i.e., when the model was trained using both (i) the most recent 600 days of price data from Phase Two and (ii) the synthetic data generated according to the methods disclosed and described with reference to Figures 1-9. Thus, training the models with a combination of actual data and synthetic data as in Phase Three was shown to have improved efficacy (as measured by the accuracy of the predicted stock prices) as compared to training the models with only actual data as in Phase One and Phase Two.

[0179] The performance of Phase Two using XGBoost improved by 0.70% with respect to the performance of Phase One. Phase Three experienced a further improvement of 0.49% as compared to Phase Two and 0.56% as compared to Phase One.

[0180] Since the average performance of stock price forecasting models that were trained with less data (600 days and the synthetic peer data) was better than the performance of the stock price forecasting models trained with more data (the entire history of data), the disclosed methods and processes are suitable for use in scenarios where less data (price or other data) is available which may occur in scenarios where the stock (or other security) is newer in the market or the nature of data used is sparse (e.g., fundamental information which comes out quarterly).

[0181] Thus, because traditional forecasting approaches typically rely on many years of historical data (often 10-15 years), these traditional approaches typically require a lot a of database storage when considering a large number of listed stocks such as the scenarios covered in Phases One, Two, and Three that considered the 500 stocks of the S&P 500. However, the disclosed approaches that use synthetic data in combination with actual data can use a shorter history of stock information (e.g., 2-3 years), which requires much less database storage than traditional approaches. As a result, the disclosed techniques improve the functioning and performance of the computing system(s) used for stock price forecasting since they (i) use less memory to store the data used for training stock price forecasting models as compared to traditional approaches and (ii) use less data overall to train the stock price forecasting models, thereby using fewer processing cycles and requiring a corresponding shorter amount of processing time as compared to traditional approaches.

[0182] The disclosed techniques can also be used in generation of synthetic information for other domains as well by adjusting the deviation (FLUX factor) with respect to domains.7. Portfolio Management Technique Overview

[0183] Figure 10 shows an overview of a portfolio management technique 1000 that implements aspects of a method of momentum trading using a plurality of deep reinforcement learning models according to some embodiments.

[0184] The portfolio management technique 1000 includes compiling and / or maintaining a database 1002 that includes information about stocks (or other securities) traded on one or more different exchanges. The database 1002 contains data relevant to the stocks (or other securities), including, for each stock (or other security), one or more (or all) of: (i) the name of the stock, (ii) the stock symbol, (iii) a company name (where applicable), (iii) a sector classification, (iv) financial information relating to the stock, such cash flow, debt, income, equity, return on equity,earnings, profits, etc., (v) historical prices of the stock, etc. In some embodiments, the database 1002 may additionally include macroeconomic data about the broader economy, sentiment data relating to the securities obtained from articles, social media, etc., and other data suitable for training DRL based machine learning models consistent with the approaches described herein.

[0185] The portfolio management technique 1000 uses the information stored in the database 1002 for two purposes: (1) a DRL Model Training Stage 1004, and (2) a Momentum Stock Identification Stage 1006.

[0186] During the DRL Model Training Stage 1004, the different DRL models are trained on the data from the database 1002 using different DRL algorithms to generate different sets of trading strategies. As described in more detail with reference to Figure 11, training one DRL model includes using a DRL agent (e.g., DRL agent 1102 in Figure 11) to interact with the data contained in the database 1002 (referred to as environment 1104 in Figure 11) to learn DRL trading policies. In operation, the DRL agent uses a particular DRL algorithm to interact with the data contained in the database 1002. In some embodiments, each DRL model uses its own DRL agent, and each DRL agent uses a different DRL algorithm. As a result, the DRL trading policies learned by each DRL agent are different.

[0187] For example, a first DRL agent uses a first DRL algorithm to generate a first set of DRL trading policies, and a second DRL agent uses a second DRL algorithm to generate a second set of DRL trading policies. Examples of the DRL algorithms used by the first DRL agent and the second DRL agent include the Advantage Actor Critic (A2C) algorithm, the Deep Q-Networks (DQN) algorithm, and / or the Proximal Policy Optimization (PPO) algorithm. However, the DRL agents may use any other DRL algorithm now known or later developed that is suitable for use in analyzing stock data to learn DRL trading policies in a manner similar to that disclosed herein. Aspects of DRL model training performed at the DRL Model Training Stage 1004 and the trading strategies generated via DRL Model Training Stage 1004 are described in further detail herein, including with reference to Figure 11. The DRL trading policies learned by the DRL agents are then combined with one or more other rules or constraints (such as rules or constraints that limit the portfolio weight for a certain security) to generate trading strategies that will be applied during the Trading Strategy Application Stage 1010

[0188] During the Momentum Stock Identification Stage 1006, a momentum screener is employed to create a candidate subset 1008 of stocks that satisfy one or more momentumthresholds. Aspects of the momentum stock identification 1006 stage are described further herein, including in Section 1(3).

[0189] Portfolio management technique 1000 next includes a Trading Strategy Application Stage 1010. During the Trading Strategy Application Stage 1010, the trading strategies generated during the DRL Model Training Stage 1004 are applied to data associated with the stocks (or other securities) in the candidate subset 1008 generated during the Momentum Stock Identification Stage 1006. In some embodiments, the data associated with the stocks (or other securities) in the candidate subset 1008 includes data obtained from database 1002, where the data from database 1002 is data that corresponds to (or is otherwise associated with) the stocks (or other securities) in the candidate subset 1008. In some embodiments, the data associated with the stocks in the candidate subset 1008 additionally or alternatively includes data from sources other than database 1002. For example, in some embodiments, the data may additionally or alternatively include real-time market data associated with the stock (e.g., current price, bid / ask size, bid / ask spread, trading volume, etc.) from one or more exchanges on which the stock is traded. In such embodiments, the real-time market data may be provided to the DRL models during the Trading Strategy Application Stage 1010 without first storing the real-time market data in database 1002.

[0190] In operation, at the Trading Strategy Application Stage 1010, the set of trained DRL models (trained during the DRL Model Training Stage 1004) are used to generate trading signals for individual stocks (or other securities) within the candidate subset 1008 of stocks (or other securities). These trading signals include BUY, SELL, and NEUTRAL signals for individual stocks (or other securities) in the candidate subset 1008. For example, for an individual stock in the candidate subset 1008, each DRL model (of the plurality of trained DRL models) generates a trading signal (e.g., buy, sell, or neutral) for the individual stock based on the data associated with that individual stock.

[0191] In some embodiments, the Trading Strategy Application Stage 1010 also includes, for an individual stock in the candidate subset 1008, combining the trading signals generated by each DRL model of the plurality of DRL models that that individual stock into a combined trading signal for that individual stock. In some embodiments, the combined trading signal has a corresponding signal strength based on the different trading signals that were combined to generate the combined trading signal. This combined trading signal is sometimes referred toherein as an “ensembled trading signal” or an “aggregated trading signal.” Generating the combined trading signal is sometimes referred to herein as “ensembling” the individual trading signals into the “ensembled trading signal” or “aggregating” the individual trading signals into the “aggregated trading signal.”

[0192] In some embodiments, the signal strength of an ensembled trading signal is obtained by averaging the separate trading signals for the stock that were generated by the plurality of DRL models. Techniques for generating the ensembled trading signal and the corresponding signal strength of an ensembled trading signal are described in more detail herein, including Section 1(4.5).

[0193] Based on these trading signals (e.g., the ensembled trading signals and / or perhaps the individual trading signals on which they are based), a portfolio 1014 of long and short positions is created at the Position Creation Stage 1012.

[0194] For example, in some embodiments, if the ensemble (or aggregated) trading signal for a first stock (or other security) generated at the Trading Strategy Application Stage 1010 is a BUY, then at the Position Creation Stage 1012, one or more shares of the first stock (or other security) are purchased and added to the portfolio 1014. Buying one or more shares of the first stock (or other security) is known as opening a long position in the first stock. Similarly, in some embodiments, if the ensemble trading signal for a second stock (or other security) is a SELL, then at the Position Creation Stage 1012, one or more shares of the second stock (or other security) are shorted, and the short position is added to the portfolio 1014. Shorting one or more shares of the second stock (or other security) is known as opening a short position in the second stock.

[0195] In some embodiments, the portfolio management technique 1000 is implemented as a long-short strategy. However, the portfolio management technique 1000 is flexible and can be readily adapted to a long-only strategy by simply considering only the long (i.e., BUY) signals for opening new long positions, or readily adapted to a short-only strategy by considering only the short (i.e., SELL) signals for opening new short positions.

[0196] In some embodiments, stock positions (long and / or short) are added to the portfolio 1014 while also adhering to diversification criteria. For example, in addition to considering the aggregated trading signals for the stocks in the candidate subset 1008 (and their corresponding signal strengths), some embodiments additionally consider one or more diversification criteria,such as opening positions (long and / or short) for stocks from different industries to avoid overconcentration of open positions in highly correlated stocks from the same industry, restricting open positions (long and / or short) for a single stock to no more than some maximum percentage of overall portfolio value to avoid overconcentration in a single stock, and / or other diversification strategies.

[0197] Open positions for the stocks in the portfolio 1014 are managed by adjusting their weights within the portfolio 1014 and / or closing the positions in response to changes in ensemble trading signals provided by the Trading Strategy Application Stage 1010. For example, at the Close or Update Weights of Open Positions Stage 1016, the aggregated trading signals from the Trading Strategy Application Stage 1010 are evaluated to determine whether to close an open position (i.e., close a long position by selling one or more shares of a stock in the portfolio 1014, or closing a short position by buying back one or more shorted shares of a stock in the portfolio 1014).

[0198] At decision block 1018, if a particular buy or sell transaction entirely closes a position in the portfolio 1014 for a particular stock, then the portfolio management technique 1000 returns to the Position Creation Stage 1012 to create one or more new positions (typically in a different stock) based on the ensembled trading signals generated by the Trading Strategy Application Stage 1010. But if the buy or sell transaction does not entirely close a position in the portfolio 1014 for a particular stock, then the portfolio management technique 1000 continues to monitor the ensembled trading signals (and perhaps their signal strengths) for that particular stock to determine whether to further add to the position in the portfolio 1014 that stock or remove a portion of the position in the portfolio 1014 for that stock.

[0199] Existing research in Deep Reinforcement Learning (DRL) applied to stock trading primarily falls into two categories: (1) trading individual stocks and (2) portfolio weight allocation.

[0200] For example, techniques for using DRL to trade individual stocks are described in: (i) Deng, Y, Bao, F., Kong, Y, Ren, Z. and Dai, Q., 2016. “Deep direct reinforcement learning for financial signal representation and trading,” IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, Issue 3, pp. 653-664 (Feb. 15, 2016); (ii) Chen, L. and Gao, Q., “Application of deep reinforcement learning on automated stock trading,” 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), pp. 29-33(Oct. 18, 2019); (iii) Wu, X., Chen, H , Wang, J., Troiano, L., Loia, V. and Fujita, H., “Adaptive stock trading strategies with deep reinforcement learning methods,” Information Sciences 538, pp.142-158 (June 2020); and (iv) Wu, Jia et al., “Quantitative trading on stock market based on deep reinforcement learning,” 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1-8 (Jul. 14, 2019). And techniques for portfolio weight allocation are described in: (i) Jiang, Zhengyao et al., “A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem,” arXiv.org (Jun. 30, 2017); (ii) Yang, Hongyang and Liu, Xiao- Yang and Zhong, Shan and Walid, Anwar, “Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy,” Proceedings of the First ACM IntT Conference on Al in Finance, pp. 1- 8, (September 11, 2020); (iii) Soleymani, Farzan & Paquet, Eric, “Financial Portfolio Optimization with Online Deep Reinforcement Learning and Restricted Stacked Autoencoder - DeepBreath,” Expert Systems with Applications, 156: 113456 (Apr. 2020); and (iv) Park, Hyungjun & Sim, Min & Choi, Dong, “An intelligent financial portfolio trading strategy using deep Q-leaming,” Expert Systems with Applications. 158: 113573 (May 2020).

[0201] However, earlier applications of DRE to trading individual stocks and portfolio weight allocation do not consider or otherwise address a holistic process of portfolio management. In contrast to the earlier applications of DRL to trading individual stocks and portfolio weight allocation described in the above-listed literature, the portfolio management technique 1000 summarized within the context of Figure 10 and described in further detail herein encompasses several of the most important stages of portfolio management, including (i) stock picking, (ii) timely opening and closing positions, and (iii) allocation of weights within the portfolio.

[0202] For example, one important aspect of portfolio management is stock selection, a facet often overlooked in existing literature. The embodiments disclosed herein address shortcomings of existing approaches by developing generalized DRL models instead of stock specific models. These generalized models are capable of processing large amounts of stocks simultaneously and generating trading signals along with signal strengths, enabling systematic stock selection.

[0203] Further, the trading signals generated by the disclosed embodiments facilitate timely opening and closing of positions, and the signals strengths of the trading signals can be used (and in some instances are used) to determine the allocation of weights of individual stocks within theportfolio (i.e., deciding how much of an individual stock should be held in the portfolio in view of the total size or value of the portfolio).

[0204] By integrating these components, the novel combination of techniques disclosed herein offer a comprehensive portfolio management approach that enhances decision-making using DRL models across all stages of portfolio management from stock screening, trade execution, and portfolio updating based on changing market conditions. And as mentioned earlier, these approaches provide improvements in the functioning of a computing system configured to implement trading and portfolio management strategies at least in part by developing, training, and operating fewer machine learning models that, in operation, analyze a smaller set of stock data as compared to prior QT strategies. As a result, the computing systems configured to manage portfolios with the disclosed embodiments described herein require far less computing resources in terms of processing cycles to implement the disclosed embodiments as compared to computing systems configured to perform various portfolio management functions with the strategies described in the above-listed literature.8. Deep Reinforcement Learning Framework

[0205] Figure 11 shows aspects of a DRL framework 1100 for training an individual DRL model according to some embodiments. The DRL framework 1100 includes a DRL agent 1102 and an environment 1104. In operation, the DRL agent 1102 learns to make decisions by interacting with the environment 1104. In operation, the decisions learned by the DRL agent 1102 are embodied in DRL trading policies, as described in further detail herein.

[0206] Figure 11 shows aspects of training a single DRL model. In embodiments that employ two or more different DRL models, the DRL framework 1100 shown and described with reference to Figure 11 is used for each DRL model (of the multiple DRL models) employed by the various embodiments described herein.

[0207] The environment 1104 includes the context or setting within which the DRL agent 1102 operates. The environment 1104 includes stock database 1106. In some embodiments, stock database 1106 is the same as or similar to database 1002 (Figure 10). As such, the information in stock database 1106 is the same as or similar to the information in database 1002.

[0208] For example, stock database 1106 includes historical data relating to individual stocks. Historical data for an individual stock is illustrated in Figure 11 as dataset 1108. Inpractice, stock database 1106 includes datasets that include data associated with many stocks, and thus, a dataset 1108 can be extracted from the stock database 1106 for each stock in the stock database 1106, i.e., the stock database 1106 contains a plurality of datasets 1108 (one dataset for each stock). A portion 1112 of the data in dataset 1108 is sampled to generate a sample set 1110 that can be used by the DRL agent 1102 to learn DRL trading policies. This size of the sample set 1110 for an individual stock in the stock database 1106 corresponds to some duration of time. The sample set 1110 may include data from any suitable duration of time, such as one or more days, one or more weeks, one or more quarters, one or more years, and so on.

[0209] In some embodiments, the environment 1104 also includes rules, dynamics, and constraints under which the environment 1104 operates. In the context of the disclosed embodiments, the environment 1104 is or includes a simulation of financial market environment that includes factors such as asset prices, economic indicators, news events, etc. associated with the set of stocks contained within the stock database 1106.

[0210] The DRL agent 1102 is a software entity that engages in interactions with the environment 1104 according to at least one of several different DRL algorithms. The DRL agent 1102 represents the intelligent decision-maker within the DRL framework 1100. In operation, the DRL agent 1102 selects actions based on its observations of the state of the environment 1104.

[0211] One objective of the DRL agent 1102 is to learn a DRL trading policy. A DRL trading policy is a mapping from states (i.e., states of the environment 1104) to actions (i.e., opening and / or closing positions in stocks). In operation, the DRL agent 1102 seeks to learn DRL trading policies that maximize cumulative reward over time. To achieve this goal of maximizing cumulative reward over time, the DRL agent 1102 uses one or more reinforcement learning algorithms to explore the environment 1104, learn from experience, and adapt its behavior to achieve desired outcomes. Examples of reinforcement learning algorithms used by the DRL agent 1102 include, but are not limited to, the Advantage Actor Critic (A2C), Deep Q- Networks (DQN), Proximal Policy Optimization (PPO) algorithms. For example, in some embodiments, the DRL agent 1102 uses one reinforcement learning algorithm to explore the environment 1104. In other embodiments, the DRL agent 1102 may use two or more reinforcement algorithms to explore the environment 1104, perhaps by using a first reinforcement learning algorithm for a first period of time to learn one or more trading policies,and then using a second reinforcement learning algorithm for a second period of time to learn different trading policies. In still further embodiments, the DRL agent 1102 may execute multiple reinforcement learning algorithms in parallel to learn different sets of trading policies based on each reinforcement learning algorithm.

[0212] In the context of some disclosed embodiments, the DRL agent 1102 is or functions as a trading bot that interacts with the financial market environment 1104 by making decisions such as buying or selling stocks based on observed market conditions. In operation, the DRL agent 1102 tracks the success and failure of the trades to learn the DRL trading policies. The trading strategies applied to market data for the securities in the candidate subset 1008 (Figure 10) during the Trading Strategy Application Stage 1010 (Figure 10) are based in whole or in part on the DRL trading policies learned by the DRL agent 1102.

[0213] In the DRL framework 1100, the interaction between the DRL agent 1102 and the environment 1104 is mathematically described using a Markov Decision Processes (MDP), defined by a tuple (S,A, P, R), where S is the state space, A is the action space, P is the state transition probability function, and R is the reward function.

[0214] The state space S represents the set of all possible states that the environment 1104 can be in. The state captures the current representation of the environment 1104, which encapsulates all (or substantially all) of the relevant information used by the DRL agent 1102 for decision-making. Decision making in the context of the disclosed embodiments includes deciding whether to open or close a long or short position in a stock.

[0215] The action space A consists of all possible actions that the DRL agent 1102 can take in the environment 1104. In the context of the disclosed embodiments, the possible actions that the DRL agent 1102 can take include, but are not limited to, one or more (or all) of (i) opening a long position (i.e., buying a stock, sometimes referred to as buying to open), (ii) closing a long position (i.e., selling a stock, sometimes referred to as selling to close), (iii) opening a short position (i.e., selling a stock short, sometimes referred to as selling to open), and (iv) closing a short position (i.e., buying back a shorted stock, sometimes referred to as buying to close). In some embodiments, the possible actions may additionally include taking no action for a particular stock, such as (i) holding a long position (i.e., deciding not to sell a stock), (ii) holding a short position (i.e., deciding not to buy back a shorted stock), (iii) deciding not to open a long position (i.e., deciding not to buy a stock), and / or (iv) deciding not to open a short position (i.e.,deciding not to sell a stock short). Tn some examples, taking no action for a particular stocks is the result of having a “neutral” outlook on the stock rather than a “buy” or “sell” outlook on the stock.

[0216] The state transition probability function P defines the probability of transitioning from one state to another state given a particular action. In the context of the disclosed embodiments, P(s' |s, ) is the probability of transitioning to state s' given the current state s and the DRL agent’s 1102 action a.

[0217] The reward function R defines the reward the DRL agent 1102 receives after taking a particular action a when the environment 1104 is in a specific state s. In some examples, the reward for opening a long position in a stock (i.e., buying the stock) is positive if the value of the stock increases after opening the long position. Similarly, the reward for opening the long position in the stock is negative if the value of the stock declines after opening the long position. Likewise, the reward for opening a short position in the stock (i.e., selling the stock short) is positive if the value of the stock declines after opening the short position, and the reward for opening the short position in the stock is negative if the value of the stock increases after opening the short position.

[0218] In the Markov decision process, the objective of the DRL agent 1102 is to find a decision making policy (i.e., a DRL trading policy) for the DRL agent 1102 so that the rewards that the DRL agent 1102 realizes from its actions increase over time, and preferably are maximized over time. But because it can be difficult to determine whether rewards have been maximized, some embodiment focus on learning DRL trading policies that increase rewards over time.

[0219] The process of maximizing (or at least increasing) rewards over time can be characterized in the context of episodes, cumulative rewards, and trading policies.

[0220] An episode refers to a single run or sequence of interactions between the DRL agent 1102 and the environment 1104, starting from an initial state of the environment 1104 and ending when a termination condition is met. In the context of the disclosed embodiments, an episode is a fixed trading period between a starting date (or time) and an end date (or time). This trading period can be any suitable duration of time, such as one or more minutes, one or more hours, one or more days, one or more weeks, one or more months, one or more quarters, one or more years, and so on.

[0221] Cumulative reward refers to the aggregated sum of rewards obtained by the DRL agent 1102 over the course of an episode.

[0222] The DRL trading policy (sometimes referred to as just a policy) TI defines the DRL agent’s 1102 strategy for selecting actions in different states. In the context of the disclosed embodiments, the DRL trading policy is a mapping from states to actions where, 7i(s) = a. In operation, TT(S) = a means that action a is selected by the DRL trading policy TT when state s exists in the environment 1104.

[0223] In operation, one objective of the DRL agent 1102 is to learn a DRL trading policy TT* that maximizes the expected cumulative reward, or at least causes the expected cumulative reward to increase over time. In operation, the DRL agent 1102 may learn many different DRL trading policies by taking different trading actions based on its observations of the environment 1104 and monitoring the rewards generated by those trading actions over time.

[0224] As described within the context of the portfolio management technique 1000 of Figure 10, and explained in further detail herein, some embodiments include multiple DRL agents 1102, where each DRL agent 1102 has used a different DRL algorithm to learn a different set of DRL trading policies from the environment 1104. These different DRL trading policies form the basis for different trading strategies implemented at the Trading Strategy Application Stage 1010.

[0225] As mentioned previously, the DRL trading policies learned by the DRL agents may be combined with each other and / or with other constraints or rules to form a trading strategy. For example, a DRL agent may learn a DRL trading policy that suggests a particular action (e g., opening a long position in a stock) based on some state (e.g., a combination of conditions relating to prior stock prices, technical information about the stock, fundamental information about the stock, broader macroeconomic conditions, technical and / or fundamental information of related stocks, and / or sentiment from publications, social media, etc.). In some embodiments, a trading strategy corresponds to a DRL trading policy in combination with one or more (i) other DRL trading policies and / or (ii) other rules and / or constraints that are separate from the policies learned by the DRL agent. For example, in some embodiments, a portfolio manager may place limits on the portfolio weight of an individual stock, or limits on how many stocks within the same industry should be held in the portfolio. Or for diversification purposes, the portfolio may be required to hold stocks from companies in one or more specific industries. The DRL tradingpolicies learned by the DRL agents can be combined with these additional rules and / or constraints to generate the trading strategies that are applied during the Trading Strategy Application Stage 1010. In some embodiments, and as described further herein, additional portfolio management driven constraints may be further applied when considering whether to open a new long position, increase or reduce an existing long position, close an existing long position, open a new short position, increase or reduce an existing short position, and / or close an existing short position.9. Identifying High Momentum Stocks

[0226] As mentioned earlier, one aspect of the portfolio management technique 1000 of Figure 10 includes the Momentum Stock Identification Stage 1006 where a momentum screener is employed to create a candidate subset 1008 (Figure 10) of stocks that satisfy one or more momentum thresholds.

[0227] In operation, a database of stocks (e.g., database 1002 from Figure 10) traded on one or more exchanges is created by gathering data on stocks, including their symbols, company names, sector classifications, financial information, historical prices and volume. This database of stocks is regularly updated with new data and also new stock listings.

[0228] As described earlier, aspects of the disclosed embodiments include identifying a list of high-momentum candidate stocks (e.g., candidate subset 1008) from the database (e.g., database 1002). In some embodiments, identifying the list of high-momentum candidate stocks is done by assigning a momentum score to each stock in the database (or to at least some stocks in the database).

[0229] In some embodiments, a total momentum score (sometimes referred to herein simply as the “momentum score” for the stock) is computed by combining both a fast momentum score and a slow momentum score for the stock.

[0230] The fast momentum score for a stock focuses on short-term price movements and trends of the stock. In operation, determining the fast momentum score for a stock involves calculating returns for the stock over a relatively shorter time period, for example, one or more hours, one or more days, one or more weeks, or one month.

[0231] By contrast, the slow momentum score for a stock looks at longer-term price movements and trends of the stock. In operation, determining the slow momentum score for astock involves calculating returns for the stock over relatively longer time horizons, such as multiple months, one or more quarters, or perhaps one or more years.

[0232] In some embodiments, the price returns for a stock are computed by excluding the most recent month’s performance. To adjust for risk, some embodiments also include dividing both the fast momentum score for the stock and the slow momentum score for the stock by a volatility factor for the stock. The volatility factor for the stock is determined by computing the annualized standard deviation of daily returns for the stock over the past few years, such as the past two or three years. The daily return for the stock on a particular day is equal to that day’s closing price divided by the previous day’s closing price minus 1, i.e., daily return = (today’s closing price / yesterday’s closing price) - 1.

[0233] In some embodiments, the fast momentum score for a stock is calculated according to Equation 1.1 month return (excluding recent 1 month) Fast momentum = - —— — — -VolatilityEquation 1

[0234] And in some embodiments, the slow momentum score for a stock is calculated according to Equation 2.6 month return (excluding recent 1 month) Slow momentum = - —— — — -V olatihtyEquation 2

[0235] In some embodiments, the risk-adjusted price momentums of the stocks in the database for both time horizons are then standardized into z-scores by subtracting the mean and then dividing the standard deviation of momentums of all stocks in the database according to Equation 3.

[0236] The final momentum score for the stock in some embodiments is the evenly weighed combination of the two z-scores calculated according to Equation 4.Equation 4

[0237] After computing the final momentum score for each stock (sometimes referred to herein as simply the “momentum score” for the stock) in the database, the stocks are ranked based on their momentum scores. These momentum scores are used to determine which stocks are added to the candidate subset 1008 (Figure 10) for subsequent analysis at the Trading Strategy Application Stage 1010 (Figure 10).

[0238] In some embodiments, stocks with higher momentum scores (e.g., momentum scores above a first threshold score) are added to the candidate subset 1008 as candidate stocks for a long position. Some embodiments additionally or alternatively include adding stocks with lower momentum scores (e.g., momentum scores below a second threshold score) to the candidate subset 1008 as candidates for a short position.

[0239] For example, in some embodiments, the momentum score for a stock takes a value between -1 and 1. In some examples, stocks having a momentum score above 0.75 are candidate stocks for a long position, and stocks having a momentum score below -0.75 are candidate stocks for a short position. Other examples may use different values than 0.75 for the first threshold (the long threshold) or -0.75 for the second threshold (the short threshold). Further, other examples may use different ranges for the momentum scores (e.g., -100 to +100) or other suitable ranges. And other thresholds suitable for use with such different ranges may be employed as well.

[0240] In some embodiments, the momentum score may not have predefined range. In such embodiments, a percentile can be used as a threshold. For example, the momentum score computed via Equation 4 will not have a set, predefined range. However, all of the positive momentum scores calculated via Equation 4 can be sorted by their percentile, and stocks with the highest positive momentum scores (e.g., in the top 75thpercentile or the top 90thpercentile) are designated as candidate stocks for a long position. And all of the negative momentum scores calculated via Equation 4 can be sorted by their percentile, and stocks with the highest negativemomentum scores (e.g., in the top 75thpercentile or the top 90thpercentile) are designated as candidate stocks for a short position. In such configurations, the percentiles function as thresholds that are used for designating candidate stocks for long and / or short positions.

[0241] For diversification purposes, some embodiments also include grouping the stocks in the database (e g., database 1002) into two or more industry and / or sector groups, and then, for each industry / sector group, (i) adding stocks in the industry / sector group having higher momentum scores (e.g., momentum scores above a first threshold score) to the candidate subset 1008 as candidate stocks for a long position for that industry / sector group, and / or (ii) adding stocks in the industry / sector group having lower momentum scores (e.g., momentum scores below a second threshold score) to the candidate subset 1008 as candidate stocks for a short position for that industry / sector group.

[0242] In some embodiments, the candidate subset 1008 is updated on a regular or semiregular basis to reflect changing market conditions. For example, a first stock may have been previously added to the candidate subset 1008 as a candidate stock for a long position because the momentum score for that first stock was above the first threshold score. But if the momentum score for that first stock later falls below the first threshold score, some embodiments include removing that first stock from the candidate subset 1008 (or even removing the first stock from the portfolio 1014 altogether). Further, if the momentum score for that first stock later falls below the second threshold score, some embodiments include adding that first stock back to the candidate subset 1008, but as a candidate stock for a short position rather than a candidate stock for a long position. In this manner, individual stocks may be added and removed from the candidate subset 1008 as candidate stocks for long and / or short positions as the momentum scores for the stocks change over time.10. Training DRL Models and Generating Trading Signals

[0243] As mentioned previously, the disclosed embodiments include generating trading signals based on the application of trading strategies (e.g., at the Trading Strategy Application Stage 1010) to current (e.g., real time or substantially real time) market data for securities in the candidate subset 1008, where the trading strategies are based at least in part on DRL trading policies learned by a DRL agent 1102 that uses one or more DRL algorithms to interact with anenvironment 1104 containing historical market data. Aspects of training the DRL models and using the trained DRL models to generate trading signals are described in this section.10.1 Training Data

[0244] The embodiments described herein employ DRL models that rely on deep neural networks. Deep neural networks rely of thousands of parameters and demand enormous amounts of data for effective training. As such, the availability and quality of training data is important for achieving high-performance models.

[0245] However, the quantity and quality of data available can vary for individual stocks. Sometimes, there is not sufficient data for training robust models independently for each stock. This may be especially true for newer companies with very limited historical data available.

[0246] For situations where one or more stocks lack sufficient historical data for use in training, some embodiments employ a cross-sectional approach where, instead of developing separate models for each stock (which is typical in prior approaches), a universal model is trained on a unified dataset that includes data for all stocks in the database. Aggregating the data into a unified dataset in this manner bolsters the amount of data available for use in training the DRL model. A DRL model that has been trained with the combined data, i.e., the unified dataset, (instead of individual stock data separately) consistent with the disclosed embodiments is able to generalize across many different stocks and learn patterns and relationships common across various stocks.10.2 State Space

[0247] The state space includes a variety of different features to represent current market conditions. In practice, a particular “state” corresponds to a combination of many features that having specific values or values within specific ranges. In some embodiments, a particular state may correspond to a combination of hundreds or even thousands of different features having various corresponding values. Aspects of some of the features are described below. However, the features described below are just examples of some of the features that can be analyzed. Some embodiments may additionally or alternatively use other features not described in detail herein.10.2.1 Technical Features

[0248] Technical features for an individual stock include indicators derived from historical prices and trading volume. Examples of technical features include, but are not limited to, exponential moving averages (EMA), relative strength index (RSI), Bollinger Bands, MACD (Moving Average Convergence Divergence), etc. These and other examples of technical features are described in Murphy, John J., “Technical analysis of the financial markets: A comprehensive guide to trading methods and applications,” Penguin (1999). In operation, the disclosed embodiments can use any technical feature(s) now known or later developed that is / are suitable for use in training a DRL model to develop DRL trading policies.10.2.2 Macroeconomic Features

[0249] Macroeconomic features include broader economic and market-related variables. Examples of macroeconomic features include, but are not limited to, Gross Domestic Product (GDP) growth, inflation rate, unemployment rate, interest rates, commodity prices (e.g., oil prices), currency exchange rates, bond yields (e.g., 2-year and 10-year U.S. Treasury bond yield), market indices (e.g., S&P 500), volatility indices (e.g., VIX), etc. In operation, the disclosed embodiments can use any macroeconomic feature(s) now known or later developed that is / are suitable for use in training a DRL model to develop DRL trading policies.10.2.3 Fundamental Features

[0250] Fundamental features include and / or relate to the underlying financial health and performance of individual companies. Examples of fundamental features include, but are not limited to, revenue growth rate, return on equity (ROE), debt-to-equity ratio, price-to-earnings (PE) ratio, price-to-book (PB) ratio, price-to-earnings growth (PEG) ratio, dividend yields, etc. In operation, the disclosed embodiments can use any fundamental feature(s) now known or later developed that is / are suitable for use in training a DRL model to develop DRL trading policies.10.2.4 Sentiment Features

[0251] Sentiment features include features derived from various information sources, such as news articles, social media posts, analyst opinions, etc. Some embodiments employ natural language processing (NLP) techniques to preprocess textual data from the information sources.In some embodiments, the NLP techniques include splitting input text into individual words or subwords called tokens. After generating the tokens, each token is converted into a numerical representation called an embedding. The token embeddings are fed into a pre-trained transformer model, such as a Bidirectional Encoder Representations from Transformers (BERT) model, which outputs a sentiment score. Aspects of using a BERT model to generate and output the sentiment score are described in Tunstall, L., Von Werra, L. and Wolf, T., “Natural language processing with transformers,” O’Reilly Media, Inc. (2022). In operation, the disclosed embodiments can use any type of NPL technique now known or later developed that is suitable for determining sentiment scores from natural language inputs that can be used in training a DRL model to develop DRL trading policies.10.2.5 Options Features

[0252] Options features include metrics such as implied volatility, open interest, put / call ratio, etc. In operation, the disclosed embodiments can use any option feature(s) now known or later developed that is / are suitable for use in training a DRL model to develop DRL trading policies.10.3 Action Space

[0253] The action space represents the possible trading decisions available to the DRL agent 1102. In some embodiments, the trading action is modeled as a discrete variable which can take three values {-1, 0, 1 }, where -1 signifies SELL, 0 indicates NEUTRAL, and 1 denotes BUY. Some embodiments may include more or fewer actions. For example, some embodiments may utilize only a buy action and a sell action. Some embodiments may implement more complicated actions, such as open a long position, close a long position, open a short position, and close a short position. Some embodiments may additionally or alternatively implement combinations of actions, such as opening a long position and buying a put option to protect the long position, or opening a short position and buying a call option to protect the short position. The action space may additionally or alternatively include any other one or more actions or combinations of actions now known or later developed that are suitable for managing a portfolio of stocks and / or other securities.10.4. Training the DRL Agent

[0254] As mentioned earlier, Figure 11 shows aspects of a Deep Reinforcement Learning (DRL) framework 1100 for training an individual DRL model according to some embodiments. In operation, aspects of using a DRL agent 1102 to train the DRL model are conducted during an episode. As explained earlier, an episode refers to a single run or sequence of interactions between the DRL agent 1102 and the environment 1104, starting from an initial state of the environment 1104 and ending when a termination condition is met. In the context of some embodiments, an episode is a fixed trading period between a starting date (and / or time) and an end date (and / or time). This trading period can be any suitable duration of time, such as one or more minutes, one or more hours, one or more days, one or more weeks, one or more months, one or more quarters, one or more years, and so on.

[0255] During training of the DRL model with the DRL agent 1102, at the start of each episode, a stock from the stock database 1106 is sampled (sometimes randomly sampled) from the database 1106 to obtain a dataset 1108 for that individual stock. As mentioned earlier, the stock database 1106 contains a plurality of datasets 1108, including at least one dataset for each stock in the stock database 1106. A portion 1112 of the data in a dataset 1108 is sampled to generate a sample set 1110 with which the DRL agent 1102 interacts during the episode. This size of the sample set 1110 for an individual stock in the stock database 1106 corresponds to some duration of time 1112. The sample set 1110 may include data from any suitable duration of time 1112, such as one or more days, one or more weeks, one or more quarters, one or more years, and so on. In some embodiments, the duration of time 1112 is a one year period data sampled from the dataset 1108 available for that stock in the stock database 1106.

[0256] During each episode, the DRL agent 1102 interacts with the environment 1104 (e.g., the sample sets 1110 from the environment 1104) by observing the state of the environment 1104, selecting trading actions based on a current DRL trading policy, and receiving rewards in terms of profit or loss resulting from the trading action.

[0257] This process of executing trades (according to the trading policy) based on the state of the environment 1104 collects experiences in the form of state-action-reward-next state tuples. Then batches of these experiences are used to update the DRL trading policy parameters of the neural network of the DRL agent 1102 through b ackpropagation to improve the current DRL trading policy. This approach allows the DRL agent 1102 to learn and improve upon DRL tradingpolicies over time to develop trading strategies across different stocks and time periods, thereby promoting generalization and adaptability.10.5 Generating Trading Signals and Signal Strengths

[0258] In some embodiments, the DRL model is trained by using one or more different DRL algorithms such as the Advantage Actor Critic (A2C), Deep Q-Networks (DQN), Proximal Policy Optimization (PPO) algorithms with the DRL agent 1102. In operation, the DRL agent 1102 can use any DRL algorithm now known or later developed that is suitable for learning, generating, or otherwise developing DRL trading policies and / or trading strategies based thereupon.

[0259] In some embodiments, (i) a first DRL agent uses a first DRL algorithm (e.g., the A2C algorithm) to interact with the environment 1104 and generate (or learn) a first set of trading strategies, (ii) a second DRL agent uses a second DRL algorithm (e.g., the DQN algorithm) to interact with the environment 1104 and generate (or learn) a second set of trading strategies, and (iii) a third DRL agent uses a third DRL algorithm (e.g., the PPO algorithm) to interact with the environment 1104 and generate (or learn) a third set of trading strategies.

[0260] In some embodiments, a single DRL agent may use multiple DRL algorithms. For example, the single DRL agent may use (i) a first DRL algorithm (e.g., the A2C algorithm) to interact with the environment and generate (or learn) a first set of trading strategies, (ii) a second DRL algorithm (e.g., the DQN algorithm) to interact with the environment and generate (or learn) a second set of trading strategies, and (iii) a third DRL algorithm (e.g., the PPO algorithm) to interact with the environment and generate (or learn) a third set of trading strategies.

[0261] In some embodiments, a DRL model implementing a trading strategy is configured to generate a trading signal (e.g., a BUY signal, a SELL signal, and / or a NEUTRAL signal) in response to detecting a particular state. In some instances, the state corresponds to a complex combination of tens, hundreds, thousands, or more different features, including but not limited to any of the technical, fundamental, macroeconomic, and / or sentiment features disclosed herein, or any other feature now known or later developed that is suitable for use in generating trading signals consistent with a trading strategy.

[0262] The BUY-NEUTRAL-SELL signals generated from the trained DRL models (sometimes referred to herein as DRL based machine learning models, or DRL models) are combined through an ensemble method, such as the ensembling processes described earlier.

[0263] In some embodiments, during the ensembling processes, each BUY, NEUTRAL, or SELL signal is associated with numbers 1, 0, and -1, respectively. The ensembled signal is obtained by taking the mean of the separate trading signals produced by all the individual DRL models. The ensemble trading signal (e.g., the mean of the separate trading signals) is interpreted as: (i) a BUY signal if the ensemble trading signal is positive; (ii) a NEUTRAL if the ensemble trading signal is equal to zero; and (iii) a SELL if the ensemble trading signal is negative.

[0264] For example, where five DRL models generated a separate trading signal for Apple stock (ticker symbol AAPL), where three of the five models generated a BUY signal, one of the five models generates a NEUTRAL signal, and one of the five models generated a SELL signal, then the final ensembled signal is calculated according to= 0.4. This ensembled signal of 0.4 is interpreted as a BUY signal for AAPL.

[0265] In operation, this ensemble process not only aggregates multiple trading signals from different DRL models, but the aggregated trading signal also provides a measure of signal strength. For example, the signal strength in some embodiments is the absolute value of the ensemble (or aggregated) trading signal, or perhaps based at least in part on the absolute value of the ensemble (or aggregated) signal.

[0266] As an example, if the ensemble SELL signal for a first stock has a value -0.8, and the ensemble SELL signal for a second stock has a value of -0.4, then the SELL signal for the first stock is stronger than the SELL signal for the second stock because |-0.8| > |-0.4|. Similarly, if the ensemble BUY signal for a third stock has a value of 0.4 and the ensemble BUY signal for a fourth stock has a value of 0.6, then the fourth stock has a stronger BUY signal than the third stock because |0.6| > |0.4|.

[0267] In some embodiments, stocks with stronger trading signals will be given preference for inclusion in the portfolio over stocks having weaker trading signals. In some instances, when a position is created in the portfolio, the weight of that position in the portfolio is (or at least can be) determined by the trading signal strength of the stock corresponding to the position.

[0268] For example, when opening a long position for a stock in the portfolio, the number of shares purchased when opening the long position may be based at least in part on the signalstrength of the BUY trading signal for that stock. For instance, if the ensemble BUY signal for a first stock has a value of 0.4 and the ensemble BUY signal for a second stock has a value of 0.6, then the second stock has a stronger BUY signal than the first stock. Thus, some embodiments may including opening a long position in both the first stock and the second stock, where the value of the long position in the second stock (i.e., number shares multiplied by the price per share) is greater than the value of the long position in the first stock.11. Portfolio Management Using Trading Signals

[0269] In some embodiments, the candidate subset 1008 (Figure 10) of high momentum stocks is generated on a regular or semi-regular basis. For example, the candidate subset 1008 may be updated on a daily, weekly, monthly, bi-monthly, quarterly, or other basis.

[0270] In some embodiments, long and / or short positions for individual stocks are created in the portfolio 1014 (Figure 10) based on the trading signals generated by a plurality DRL models (i.e., a set of two or more DRL models) during the Trading Strategy Application Stage 1010 (Figure 10) for the stocks in the candidate subset 1008 (Figure 10).

[0271] In some instances, portfolio weights are allocated to individual positions based on the signal strengths of the trading signals (BUY, SELL, etc.). In some embodiments, the trading signals during the Trading Strategy Application Stage 1010 are generated on a regular or semiregular basis (e.g., hourly, every other hour or few hours, daily, every other day or few days, weekly, etc.) using the trading strategies based on the DRL trading policies generated (i.e., learned) by the DRL agents 1102 (Figure 11). The portfolio weight for individual stocks in some embodiments based at least in part on the trading signal strengths.

[0272] If there is a change in the trading signal strength for an individual stock, the weight for that stock in the portfolio 1014 is (or at least can be) updated accordingly. In some embodiments, updating the portfolio weight for a particular stock includes, for example, any one or more (or all) of (i) adding more shares of the stock to a long position (i.e., buying more shares) when the trading signal strength for that stock increases, (ii) removing some (but perhaps not all) shares of a stock from a long position (i.e., selling some shares) when the trading signal strength for that stock declines, (iii) adding more shares of the stock to a short position (i.e., shoring more shares) when the trading signal strength for that stock declines, and / or (iv)removing some (but perhaps not all) shares of the stock from a short position (i.e., buying back some shares) when the trading signal strength for that stock increases.

[0273] In some instances, if there is a complete reversal in the trading signal strength for a specific stock, the corresponding position for that stock is closed. For example, if the portfolio includes a long position in a first stock, and the trading signal for that first stock changes from a positive (BUY) value to a negative (SELL) value, some embodiments include selling all of the shares in the long position (i.e., closing the long position) for that first stock. Similarly, if the portfolio includes a short position in a second stock, and the trading signal for that second stock changes from a negative (SELL) value to a positive (BUY) value, some embodiments include buying back all of the shares in the short position (i.e., closing the short position) for that second stock.

[0274] A position can also be closed at any time if a stop-loss is triggered. For example, when the portfolio has a long position in a first stock where the current price of the first stock is $10 per share, and where the portfolio is configured with a stop order for the first stock at $8 per share, some embodiments include closing the long position in that first stock (i.e., selling all the shares of that first stock in the position) in response to the price falling to $8 per share even if the trading signal for that first stock may still have a positive (BUY) value. Similarly, when the portfolio has a short position in a second stock where the current price of the second stock is $10 per share, and where the portfolio is configured with a stop order for the second stock at $12 per share, some embodiments include closing the short position in that second stock (i.e., buying back all the shares of that second stock in the position) in response to the price rising to $12 per share even if the trading signal for that second stock may still have a negative (SELL) value.

[0275] In some embodiments, when a position is closed, a new position (with a different stock) is added as a replacement for the recently-closed position. In operation, the new stock is selected from the candidate subset 1008 based at least in part on the signal strength of the trading signals of the stocks in the candidate subset 1008.

[0276] In some embodiments, opening new positions, closing existing positions, and / or adjusting the positions happens when the candidate subset 1008 is updated, which may include adding and / or removing stocks from the candidate subset 1008 and / or updating the trading signals for the stocks in the candidate subset 1008.

[0277] Additional aspects of the three main steps, i.e., opening positions, adjusting the weight allocation of positions, and closing positions, are described below.11.1 Adding Shares of Stocks to the Portfolio

[0278] As explained above, stocks are added to the portfolio 1014 (Figure 10) based on the BUY (for long positions) and / or SELL (for short positions) signals generated by the ensembled trading signals generated by the DRL models during the Trading Strategy Application Stage 1010 (Figure 10).

[0279] In some instances, only BUY signals on positive momentum stocks in the candidate subset 1008 and SELL signals on negative momentum stocks in the candidate subset 1008 are considered when opening new long and short positions, respectively. Preference is given to stocks with higher signal strength, as described above, indicating greater potential for favorable returns.

[0280] In some embodiments, and as described earlier, stocks are included in the portfolio 1014 additionally based on a diversification criteria that aims to incorporate stocks from different industries and / or avoid an overconcentration of an individual stock or a set of highly correlated stocks from the same industry.11.2 Weight Assignment within the Portfolio

[0281] In some embodiments, weights of the positions in the portfolio 1014 are determined based on the signal strengths of the trading signals generated by the set of one or more DRL agents 1102.

[0282] The weight of a position corresponds at least in part to the value of the position as a percentage of the total value of the portfolio. In one example, if the total value of the portfolio 1014 is $50MM, and the total value of a first long position for a first stock is S5MM, then the weight of that first stock in the portfolio 1014 is 10% (i.e., S5MM / S50MM). In another example where the total value of the portfolio 1014 is S50MM, and the total value of a second long position for a second stock is SIMM, the weight of that second stock in the portfolio 1014 is 2% (i.e., SIMM / S50MM). Portfolio weights could be calculated by other metrics as well.

[0283] In some embodiments, portfolio weights for individual stocks in the portfolio 1014 are calculated by applying the softmax function to the ensembled signal strengths generatedduring the Trading Strategy Application Stage 1010 (Figure 10). The softmax function takes as input a vector z of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponentials of the input numbers. Aspects of applying the softmax function are described by Goodfellow, Ian et al., “6.2.2.3 Softmax Units for Multinoulli Output Distributions,” Deep Learning, pp. 180-184, MIT Press, ISBN 978-0-26203561-3 (2016).

[0284] In operation, softmax converts the raw trading signal strengths to numbers that add up to 1 where each number represents a target weight of the corresponding stock in the portfolio. In some embodiments, the softmax expression for the weight of the jth constituent stock in the portfolio 1014 is calculated according to Equation 5.Equation 5 where the s1;sn, are signal strengths of portfolio constituents.

[0285] Using the softmax function in this manner ensures that the weights for each stock in the portfolio 1014 are proportional to their respective trading signal strengths. In operation, the allocation of the stocks in the portfolio 1014 can be updated (and in some instances is updated) based on the calculated weights so that stocks with stronger trading signals can have greater weights in the overall portfolio 1014 than stocks with weaker trading signals.11.3 Removing Shares of Stocks from the Portfolio

[0286] In some embodiments, a stock is removed from the portfolio 1014 in response to a trading signal reversal for the stock.

[0287] For example, and as described earlier, a long position may be closed (and the shares of that long position removed from the portfolio) when the trading signal for the stock associated with the long position changes from a BUY signal to a SELL signal. Similarly, a short position may be closed (and the shorted shares for that short position removed from the portfolio) when the trading signal the stock associated with the short position change from a SELL signal to a BUY signal.

[0288] Additionally, in some embodiments, stocks can be removed from the portfolio if they trigger stop-loss.12. Example Methods for Momentum Trading with DRL Algorithms

[0289] Figure 12 shows a flow chart method 1200 for momentum trading using a plurality of deep reinforcement learning models according to some embodiments.

[0290] In operation, method 1200 is implemented via a computing system, such as computing system 1300 shown and described with reference to Figure 13, or any other type of computing system now known or later developed that is suitable for performing the computing functions disclosed and described herein.

[0291] Method 1200 begins at block 1202, which includes, for a first Deep Reinforcement Learning (DRL) model in a plurality of DRL models, generating a first plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. In some embodiments, the one or more technical features corresponding to individual securities in a set of securities, fundamental features of individual securities in the set of securities, and macroeconomic data are stored in a database such as database 1002 (Figure 10) and / or database 1106 (Figure 11).

[0292] In some embodiments, generating the first plurality of trading strategies for the first DRL model at block 1202 includes using a first DRL agent to learn the first plurality of trading strategies by using a first DRL algorithm to interact with an environment comprising the (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. In some examples, the first DRL algorithm comprises at least one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm. Any other suitable DRL algorithm could be used, too.

[0293] In some embodiments, using the first DRL agent to learn the first plurality of trading strategies by using the first DRL algorithm to interact with the environment includes generating at least one DRL policy that maps a state to an action, where (i) the state for the at least one DRL policy corresponds to a set of one or more (a) technical features corresponding to individual securities in the set of securities, (b) fundamental features of individual securities in the set of securities, and (c) macroeconomic data; (ii) the action for the at least one DRL policy corresponds to one of buying or selling a security; and (iii) an individual trading strategy in the first plurality of trading strategies is based at least in part on the at least one DRL policy.

[0294] At block 1204, method 1200 includes for a second DRL model in the plurality of DRL models, generating a second plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data.

[0295] In some embodiments, generating the second plurality of trading strategies for the second DRL model at block 1204 includes using a second DRL agent to learn the second plurality of trading strategies by using a second DRL algorithm to interact with the environment comprising the (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data. In some examples, the second DRL algorithm comprises at least one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm.

[0296] In some embodiments, the first DRL algorithm used in block 1202 is a different DRL algorithm than the second DRL algorithm used in block 1204. In some embodiments, the first DRL algorithm used in block 1202 is the same DRL algorithm than the second DRL algorithm used in block 1204, except that the first and second DRL algorithms in such a scenario are configured differently such that they generate different trading policies and / or trading strategies.

[0297] In some embodiments, using the second DRL agent to learn the second plurality of trading strategies by using the second DRL algorithm to interact with the environment includes generating at least one DRL policy that maps a state to an action, where (i) the state for the at least one DRL policy corresponds to a set of one or more (a) technical features corresponding to individual securities in the set of securities, (b) fundamental features of individual securities in the set of securities, and (c) macroeconomic data; (ii) the action for the at least one DRL policy corresponds to one of buying or selling a security; and (iii) an individual trading strategy in the first plurality of trading strategies is based at least in part on the at least one DRL policy.

[0298] In some embodiments, the plurality of DRL models includes additional DRL models, such as three, four, five, or more DRL models. Some such embodiments additionally include, for each additional DRL model in the plurality of DRL models, generating an additional plurality of trading strategies based at least in part on one or more (i) technical features corresponding toindividual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data

[0299] At block 1206, method 1200 includes determining a momentum score for each security in the set of securities.

[0300] In some embodiments, determining a momentum score for each security in the set of securities at block 1206 includes, for an individual security in the set of securities: (i) calculating a fast momentum score for the individual security based on a one month return of the security divided by a volatility of the security; (ii) calculating a z-score for the fast momentum score by determining a difference between a mean of momentums of all securities in the set of securities and the fast momentum score for the individual security, and dividing the difference by a standard deviation of momentums of all securities in the set of securities; (iii) calculating a slow momentum score for the individual security based on a six month return of the security divided by the volatility of the security; (iv) calculating a z-score for the slow momentum score for the individual security by determining a difference between a mean of momentums of all securities in the set of securities and the slow momentum score for the individual security, and dividing the difference by the standard deviation of momentums of all securities in the set of securities; and (v) determining the momentum score for the individual security by determining a sum of the z- score for the fast momentum for the individual security and the z-score of the slow momentum for the individual security, and dividing the sum by two to produce the momentum score for the individual security. In some such embodiments, the volatility of the individual security is determined by computing an annualized standard deviation of daily returns of the individual security over a three year period.

[0301] At block 1208, method 1200 includes selecting a candidate subset of securities from the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold.

[0302] In some embodiments, selecting a candidate subset of securities from the set of securities, at block 1208 includes at least one of (i) selecting at least one security having a momentum score greater than a first threshold for inclusion in the candidate subset of securities as a candidate for a long position; or (ii) selecting at least one security having a momentum score less than a second threshold for inclusion in the candidate subset of securities as a candidate for a short position

[0303] At block 1210, method 1200 includes for each security in the candidate subset of securities, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading strategies to data associated with the security, (ii) generating a second set of one or more trading signals for the security by applying the second plurality of trading strategies to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security, wherein the aggregated trading signal indicates a signal strength.

[0304] In some embodiments, the aggregated trading signal for an individual security generated at block 1210 is (or at least includes) a value between -1 and +1, wherein an aggregated trading signal having a positive value corresponds to a buy indication, wherein an aggregated trading signal having a negative value corresponds to a sell indication, wherein an aggregated trading signal having a value closer to +1 is a stronger buy indication than an aggregated trading signal having a positive value closer to 0, wherein an aggregated trading signal having a value closer to -1 is a stronger sell indication than an aggregated trading signal having a negative value closer to 0.

[0305] At optional block 1212, method 1200 includes determining a portfolio weight for each security in the trading portfolio, wherein for an individual security in the trading portfolio, the portfolio weight for the individual security is proportional to a strength of the aggregated trading signal corresponding to the individual security.

[0306] In some embodiments, determining a portfolio weight for each security in the trading portfolio at block 1212 includes determining the portfolio weight w for security j according to & Si ‘ the equation: Wj = where Sp ... , sn, are strengths of the aggregated trading signals for ^i=ie 1each of security / through security n in the trading portfolio.

[0307] At block 1214, method 1200 includes adding one or more securities from the candidate subset of securities to a trading portfolio based on the aggregated trading signals for the securities in the subset of securities.

[0308] In some embodiments, adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities at block 1214 includes, for an individual security, adding the individual security to the trading portfolio as a long position when (i) the value of the aggregated tradingsignal for the individual security a positive value, (ii) the strength of the aggregated trading signal for the individual security is above a threshold strength, and (iii) a number of shares of the individual security added to the trading portfolio is based on the portfolio weight of the individual security.

[0309] In some embodiments, adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities at block 1214 includes, for an individual security, adding the individual security to the trading portfolio as a short position when (i) the value of the aggregated trading signal for the individual security is a negative value, (ii) the strength of the aggregated trading signal for the individual security is below a threshold strength, and (iii) a number of shares shorted in the trading portfolio is based on the portfolio weight of the individual security.

[0310] At optional block 1216, method 1200 includes monitoring changes in the aggregate trading signals for each security in the trading portfolio, and removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security.

[0311] In some embodiments, removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security at block 1216 includes, for an individual security having a long position in the trading portfolio, removing the individual security from the trading portfolio when the aggregated trading signal changes from a positive value to a negative value. And in some embodiments, removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security at block 1216 additionally or alternatively includes, for an individual security having a short position in the trading portfolio, removing the individual security from the trading portfolio when the aggregated trading signal changes from a negative value to a positive value.13. Example Computing Systems

[0312] Figure 13 shows an example computing system 1300 configured for implementing one or more (or all) aspects of the methods and processes disclosed herein.

[0313] Computing system 1300 includes one or more processors 1302, one or more tangible, non-transitory computer-readable memory 1304, one or more user interfaces 1306, and one or more network interfaces 1308.

[0314] The one or more processors 1302 may include any type of computer processor now known or later developed that is suitable for performing one or more (or all) of the disclosed features and functions, individually or in combination with one or more additional processors

[0315] The one or more tangible, non-transitory computer-readable memory 1304 is configured to store program instructions that are executable by the one or more processors 1302. The program instructions, when executed by the one or more processors 1302 cause the computing system to perform any one or more (or all) of the functions disclosed and described herein. In operation, the one or more tangible, non-transitory computer-readable memory 1304 is also configured to store data that is both used in connection with performing the disclosed functions (e.g., pricing data used at block 902 (FIG. 9A) and / or the data for environment 1104 (FIG. 11)) and generated via performing the disclosed functions (e.g., the synthetic data at block 904 (FIG. 9A), the trading signals (FIG. 10), the portfolio information 1014 (FIG. 10) and so on).

[0316] The one or more user interfaces 1306 may include any one or more of a keyboard, monitor, touchscreen, mouse, trackpad, voice interface, or any other type of user interface now known or later developed that is suitable for receiving inputs from a computer user or another computer and / or providing outputs to a computer user or another computer.

[0317] The one or more network interfaces 1308 may include any one or more wired and / or wireless network interfaces, including but not limited to Ethernet, optical, WiFi, Bluetooth, or any other network interface now known or later developed that is suitable for enabling the computing system 1300 to receive data from other computing devices and systems and / or transmit data to other computing devices and systems.

[0318] The computing system 1300 corresponds to any one or more of a desktop computer, laptop computer, tablet computer, smartphone, and / or computer server acting individually or in combination with each other to perform the disclosed features.14. Example Embodiments

[0319] For ease of illustration, the Example Embodiments are grouped into Examples 1-20 that generally relate to features for generating synthetic trade data and Examples 21-40 thatgenerally relate to features for momentum trading using DRL algorithms. In operation, any of Examples 1-20 can be combined with any of Examples 21-40.

[0320] For example, with respect to the methods of Examples 1-13 and 21-34, the synthetic trading data generated via any of Examples 1-13 can used to train the Deep Reinforcement Learning (DRL) models used in connection with the momentum trading approaches in any of Examples 21-34. Further, any of the methods in Examples 1-13 and / or Examples 21-34, individually or combination, may be implemented by one or more computing systems. Similarly, the methods in Examples 1-13 and / or Examples 21-34, individually or in combination, may be embodied in program instructions held on tangible, non-transitory computer-readable media. Persons of skill in the art will readily recognize the combinability of the various features for use with the disclosed examples and variations thereof.14.1 Example Embodiments for Generating Synthetic Trade Data

[0321] Example 1 : A method comprising: (i) receiving an actual dataset comprising a time-ordered sequence of actual historical price data of an actual security during a timeframe, wherein the actual security is listed on a financial exchange, and wherein the time-ordered sequence of actual historical price data comprises a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe; (ii) based on the actual dataset, generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe, wherein each fictional security of the plurality of fictional securities is not listed on any financial exchange, wherein an individual synthetic dataset for an individual fictional security comprises a synthetic time-ordered sequence of historical price data for the fictional security, and wherein the time-ordered sequence of synthetic historical price data comprises a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe; and (iii) training a machine learning model with data comprising the actual dataset and the plurality of synthetic datasets, wherein after training the machine learning model, the machine learning model is configured to forecast a future price data of the actual security. In operation, the method of Example 1 may be performed by a computing system, such as the example computing system shown and described with reference to Figure 13, or any other type of configuration of computing system now known or later developed that is suitable for performing the functions disclosed herein.

[0322] Example 2: The method of Example 1, wherein generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe comprises, for an individual fictional security: (i) for each market price of the actual security at each time point of the plurality of time points during the timeframe, generating a synthetic price at the time point by multiplying the market price by a random number, wherein market prices at different time points are multiplied by different random numbers to generate the synthetic prices at the different time points; and (ii) ordering the generated synthetic prices based on their corresponding time points into a synthetic dataset for the individual fictional security.

[0323] Example 3 : The method of Example 2, wherein the random number is within a range defined by a lower bound and an upper bound.

[0324] Example 4: The method of Example 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprise multiplying the market price by a random number that is less than 1, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point.

[0325] Example 5: The method of Example 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises multiplying the market price by a random number that is greater than 1, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point.

[0326] Example 6: The method of Example 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises: (i) generating a synthetic price at the time point by multiplying the market price by a first random number based on a second random number that is different than the first random number, wherein: (a) when the second random number has a first attribute, the first random number is less than one, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point; and (b) when the second random number has a second attribute, the first random number is greater than one, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point.

[0327] Example 7: The method of any of Examples 1-6, wherein training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets comprises: (i) for the actual security, extracting one or more features from price data in the actual dataset; (ii) for the plurality of fictional securities, for each fictional security, extractingthe one or more features from price data in the synthetic dataset for the fictional security; and (iii) training the model to predict the future price of the actual security based at least in part on the one or more extracted features from the price data in the actual dataset and the one or more extracted features from the price data in each synthetic dataset.

[0328] Example 8: The method of Example 7, wherein the one or more features, comprise one or more of: (i) a Relative Strength Index (RSI) that quantifies a speed and change of price movement of the price data, (ii) a Moving Average Convergence / Di vergence (MACD) using two exponential moving averages of different timeframes to identify a strength of a directional move in the price data, (iii) an Average Direction Index (ADX) that measures a strength and momentum of change in the price data, (iv) a Stochastic Oscillator that measures a current price relative to a price range over a number of periods, (v) a Simple Moving Average (SMA) that measures an average price over a specific time period, or (vi) a Standard Deviation that measures volatility in the price data.

[0329] Example 9: The method of any of Examples 1-8, further comprising, after training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets predicting a future price for the actual security.

[0330] Example 10: The method of any of Examples 1-9, further comprising: (i) after training the machine learning model, receiving a subsequent actual dataset comprising a new time-ordered sequence of actual historical price data for the actual security during a subsequent timeframe, wherein the subsequent actual dataset includes market price data of the actual security on the financial exchange at each time point of a plurality of time points during the subsequent timeframe; (ii) based on the subsequent actual dataset, generating a subsequent synthetic dataset for each fictional security during the subsequent timeframe, wherein an individual subsequent synthetic dataset for an individual fictional security comprises a time- ordered sequence of synthetic price data for the fictional security comprising a synthetic price of the fictional security at each time point of the plurality of time points during the subsequent timeframe; (iii) generating an updated actual dataset by appending the subsequent actual dataset with at least a portion of the previously-received actual dataset; (iv) generating a plurality of updated synthetic datasets by generating an updated synthetic dataset for each fictional security, wherein generating an updated synthetic dataset for an individual fictional security comprises appending the subsequent synthetic dataset for the fictional security with at least a portion of thepreviously-generated synthetic dataset for the fictional security; and (v) re-training the machine learning model on data comprising the updated actual dataset and the plurality of updated synthetic datasets, wherein after re-training, the machine learning model is configured to forecast future price data of the actual security.

[0331] Example 11 : The method of any of Examples 1-10, wherein generating the plurality of synthetic datasets for the plurality of fictional securities during the timeframe comprises, for an individual synthetic dataset for an individual fictional security: (i) applying a bi-stage transformation to each data point Si of the actual dataset to produce a corresponding synthetic data point pi for the synthetic dataset, wherein the bi-stage transformation is defined by an equation pi = (si0i) \| / iSi wherein 0i represents a maximum deviation percentage of pi from Si, and \| / i represents an associativity factor that determines a direction of deviation that is positive or negative.

[0332] Example 12: The method of Example 11, further comprising: (i) selecting a value for 0i within a predetermined range of 0 to L, where L is a predefined maximum deviation limit, and 0i is a real number between 0 and L; and (ii) setting \| / i to a positive or negative value, wherein (i) when \| / i is positive, synthetic price pi is greater than or equal to actual price Si, and (ii) when \| / i is negative, synthetic price p; is less than or equal to actual price Si.

[0333] Example 13: The method of any of Examples 1-12, wherein the actual security comprises one of (i) a stock, (ii) a bond, (iii) a derivative, or (iv) a market index.

[0334] Example 14: Tangible, non-transitory computer-readable media comprising program instructions, wherein the program instructions, when executed by one or more processors, cause a computing system to perform functions comprising: (i) receiving an actual dataset comprising a time-ordered sequence of actual historical price data of an actual security during a timeframe, wherein the actual security is listed on a financial exchange, and wherein the time-ordered sequence of actual historical price data comprises a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe; (ii) based on the actual dataset, generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe, wherein each fictional security of the plurality of fictional securities is not listed on any financial exchange, wherein an individual synthetic dataset for an individual fictional security comprises a synthetic time-ordered sequence of historical price data for the fictional security, and wherein the time-ordered sequence ofsynthetic historical price data comprises a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe; and (iii) training a machine learning model with data comprising the actual dataset and the plurality of synthetic datasets, wherein after training the machine learning model, the machine learning model is configured to forecast a future price data of the actual security.

[0335] Example 15: The tangible, non-transitory computer-readable media of Example14, wherein generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe comprises, for an individual fictional security: (i) for each market price of the actual security at each time point of the plurality of time points during the timeframe, generating a synthetic price at the time point by multiplying the market price by a random number, wherein market prices at different time points are multiplied by different random numbers to generate the synthetic prices at the different time points; and (ii) ordering the generated synthetic prices based on their corresponding time points into a synthetic dataset for the individual fictional security.

[0336] Example 16: The tangible, non-transitory computer-readable media of Example15, wherein the random number is within a range defined by a lower bound and an upper bound, and wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises at least one of (i) multiplying the market price by a random number that is less than 1, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point, or (ii) multiplying the market price by a random number that is greater than 1, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point.

[0337] Example 17: The tangible, non-transitory computer-readable media of any ofExamples 14 through 16, wherein training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets comprises: (i) for the actual security, extracting one or more features from price data in the actual dataset; (ii) for the plurality of fictional securities, for each fictional security, extracting the one or more features from price data in the synthetic dataset for the fictional security; and (iii) training the model to predict the future price of the actual security based at least in part on the one or more extracted features from the price data in the actual dataset and the one or more extracted features from the price data in each synthetic dataset.

[0338] Example 18: The tangible, non-transitory computer-readable media of Example17, wherein the one or more features comprise one or more of: (i) a Relative Strength Index (RSI) that quantifies a speed and change of price movement of the price data, (ii) a Moving Average Convergence / Divergence (MACD) using two exponential moving averages of different timeframes to identify a strength of a directional move in the price data, (iii) an Average Direction Index (ADX) that measures a strength and momentum of change in the price data, (iv) a Stochastic Oscillator that measures a current price relative to a price range over a number of periods, (v) a Simple Moving Average (SMA) that measures an average price over a specific time period, or (vi) a Standard Deviation that measures volatility in the price data.

[0339] Example 19: The tangible, non-transitory computer-readable media of any ofExamples 14 through 18, further comprising, after training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets, predicting a future price for the actual security, wherein the actual security comprises one of (i) a stock, (ii) a bond, (iii) a derivative, or (iv) a market index.

[0340] Example 20: The tangible, non-transitory computer-readable media of any ofExamples 14 through 18, further comprising: (i) after training the machine learning model, receiving a subsequent actual dataset comprising a new time-ordered sequence of actual historical price data for the actual security during a subsequent timeframe, wherein the subsequent actual dataset includes market price data of the actual security on the financial exchange at each time point of a plurality of time points during the subsequent timeframe; (ii) based on the subsequent actual dataset, generating a subsequent synthetic dataset for each fictional security during the subsequent timeframe, wherein an individual subsequent synthetic dataset for an individual fictional security comprises a time-ordered sequence of synthetic price data for the fictional security comprising a synthetic price of the fictional security at each time point of the plurality of time points during the subsequent timeframe; (iii) generating an updated actual dataset by appending the subsequent actual dataset with at least a portion of the previously-received actual dataset; (iv) generating a plurality of updated synthetic datasets by generating an updated synthetic dataset for each fictional security, wherein generating an updated synthetic dataset for an individual fictional security comprises appending the subsequent synthetic dataset for the fictional security with at least a portion of the previously-generated synthetic dataset for the fictional security; and (v) re-training the machine learning model on datacomprising the updated actual dataset and the plurality of updated synthetic datasets, wherein after re-training, the machine learning model is configured to forecast future price data of the actual security.14.2 Example Embodiments for Momentum Trading using DRL Algorithms

[0341] Example 21 : A method performed by a computing system, the method comprising:(A) for a first Deep Reinforcement Learning (DRL) model in a plurality of DRL models, generating a first plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (B) for a second DRL model in the plurality of DRL models, generating a second plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (C) determining a momentum score for each security in the set of securities; (D) selecting a candidate subset of securities from the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold; (A) for each security in the candidate subset of securities, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading strategies to data associated with the security, (ii) generating a second set of one or more trading signals for the security by applying the second plurality of trading strategies to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security, wherein the aggregated trading signal indicates a signal strength; and (E) adding one or more securities from the candidate subset of securities to a trading portfolio based on the aggregated trading signals for the securities in the subset of securities.

[0342] Example 22: The method of Example 21, further comprising: determining a portfolio weight for each security in the trading portfolio, wherein for an individual security in the trading portfolio, the portfolio weight for the individual security is proportional to a strength of the aggregated trading signal corresponding to the individual security.

[0343] Example 23: The method of Example 22, wherein determining a portfolio weight for each security in the trading portfolio comprises determining the portfolio weight w fore V security j according to the equation Wj = — — where s1(... , sn, are strengths of the aggregated .i=ie 1trading signals for each of security 1 through security n in the trading portfolio.

[0344] Example 24: The method of any of Examples 21 through 23, wherein adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities comprises, for an individual security, adding the individual security to the trading portfolio as a long position when(i) the value of the aggregated trading signal for the individual security a positive value, (ii) the strength of the aggregated trading signal for the individual security is above a threshold strength, and (iii) a number of shares of the individual security added to the trading portfolio is based on the portfolio weight of the individual security.

[0345] Example 25: The method of any of Examples 21 through 24, wherein adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities comprises, for an individual security, adding the individual security to the trading portfolio as a short position when (i) the value of the aggregated trading signal for the individual security is a negative value,(ii) the strength of the aggregated trading signal for the individual security is below a threshold strength, and (iii) a number of shares shorted in the trading portfolio is based on the portfolio weight of the individual security.

[0346] Example 26: The method of any of Examples 21 through 25, wherein an aggregated trading signal for an individual security is a value between -1 and +1, wherein an aggregated trading signal having a positive value corresponds to a buy indication, wherein an aggregated trading signal having a negative value corresponds to a sell indication, wherein an aggregated trading signal having a value closer to +1 is a stronger buy indication than an aggregated trading signal having a positive value closer to 0, wherein an aggregated trading signal having a value closer to -1 is a stronger sell indication than an aggregated trading signal having a negative value closer to 0.

[0347] Example 27: The method of any of Examples 21 through 26, further comprising monitoring changes in the aggregate trading signals for each security in the trading portfolio, and removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security.

[0348] Example 28: The method of Example 27, wherein removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security comprises, for an individual security having a long position in the trading portfolio, removing the individual security from the trading portfolio when the aggregated trading signal changes from a positive value to a negative value.

[0349] Example 29: The method of Example 27, wherein removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security comprises, for an individual security having a short position in the trading portfolio, removing the individual security from the trading portfolio when the aggregated trading signal changes from a negative value to a positive value.

[0350] Example 30: The method of any of Examples 21-29, wherein: (A) generating the first plurality of trading strategies for the first DRE model comprises using a first DRL agent to learn the first plurality of trading strategies by using a first DRL algorithm to interact with an environment comprising the (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (B) generating the second plurality of trading strategies for the second DRL model comprises using a second DRL agent to learn the second plurality of trading strategies by using a second DRL algorithm to interact with the environment comprising the (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (C) wherein the first DRL algorithm comprises one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm; and (D) wherein the second DRL algorithm comprises one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm.

[0351] Example 31 : The method of any of Examples 21 through 30, wherein the step of“generating the first plurality of trading strategies for the first DRL model comprises using a first DRL agent to learn the first plurality of trading strategies by using a first DRL algorithm to interact with an environment comprising the (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data” comprises generating at least one DRL policy thatmaps a state to an action, wherein: (A) the state for the at least one DRL policy corresponds to a set of one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (B) the action for the at least one DRL policy corresponds to one of buying or selling a security; and (C) an individual trading strategy in the first plurality of trading strategies is based at least in part on the at least one DRL policy.

[0352] Example 32: The method of any of Examples 21 through 31, wherein determining a momentum score for each security in the set of securities comprises, for an individual security in the set of securities: (i) calculating a fast momentum score for the individual security based on a one month return of the security divided by a volatility of the security; (ii) calculating a z-score for the fast momentum score by determining a difference between a mean of momentums of all securities in the set of securities and the fast momentum score for the individual security, and dividing the difference by a standard deviation of momentums of all securities in the set of securities; (iii) calculating a slow momentum score for the individual security based on a six month return of the security divided by the volatility of the security; (iv) calculating a z-score for the slow momentum score for the individual security by determining a difference between a mean of momentums of all securities in the set of securities and the slow momentum score for the individual security, and dividing the difference by the standard deviation of momentums of all securities in the set of securities; and (v) determining the momentum score for the individual security by determining a sum of the z-score for the fast momentum for the individual security and the z-score of the slow momentum for the individual security, and dividing the sum by two to produce the momentum score for the individual security.

[0353] Example 33 : The method of Example 32, wherein the volatility of the individual security is determined by computing an annualized standard deviation of daily returns of the individual security over a three year period.

[0354] Example 34: The method of any of Examples 21 through 33, wherein selecting a candidate subset of securities from the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold comprises at least one of: (i) selecting at least one security having a momentum score greater than a first threshold for inclusion in the candidate subset of securities as a candidate for a longposition; or (ii) selecting at least one security having a momentum score less than a second threshold for inclusion in the candidate subset of securities as a candidate for a short position.

[0355] Example 35: Tangible, non-transitory computer-readable media having program instructions stored therein, wherein the program instructions, when executed by one or more processors, cause a computing system to perform functions comprising: (A) for a first Deep Reinforcement Learning (DRL) model in a plurality of DRL models, generating a first plurality of trading policies based at least in part on one or more (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (B) for a second DRL model in the plurality of DRL models, generating a second plurality of trading policies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (C) determining a momentum score for each security in the set of securities; (D) selecting a candidate subset of securities from the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold; (E) for each security in the candidate subset of securities, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading policies to data associated with the security, (ii) generating a second set of one or more trading signals for the security by applying the second plurality of trading policies to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security, wherein the aggregated trading signal indicates a signal strength; and (F) adding one or more securities from the candidate subset of securities to a trading portfolio based on the aggregated trading signals for the securities in the subset of securities.

[0356] Example 36: The tangible, non-transitory computer-readable media of Example35, wherein functions further comprise determining a portfolio weight for each security in the trading portfolio, wherein for an individual security in the trading portfolio, the portfolio weight for the individual security is proportional to a strength of the aggregated trading signal corresponding to the individual security, and wherein adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities comprises, for an individual security, at least one of: (A)adding the individual security to the trading portfolio as a long position when (i) the value of the aggregated trading signal for the individual security a positive value, (ii) the strength of the aggregated trading signal for the individual security is above a threshold strength, and (iii) a number of shares of the individual security added to the trading portfolio is based on the portfolio weight of the individual security; or (B) adding the individual security to the trading portfolio as a short position when (i) the value of the aggregated trading signal for the individual security is a negative value, (ii) the strength of the aggregated trading signal for the individual security is below a threshold strength, and (iii) a number of shares shorted in the trading portfolio is based on the portfolio weight of the individual security.

[0357] Example 37: The tangible, non-transitory computer-readable media of Example36, wherein the functions further comprise: (i) monitoring changes in the aggregate trading signals for each security in the trading portfolio, and (ii) removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security.

[0358] Example 38: A computing system comprising (i) one or more processors; and(ii) tangible, non-transitory computer-readable media having program instructions stored therein for execution by the one or more processors. In some embodiments, the program instructions, when executed by the one or more processors, cause the computing system to perform functions comprising: (A) for a first Deep Reinforcement Learning (DRL) model in a plurality of DRL models, generating a first plurality of trading policies based at least in part on one or more (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (B) for a second DRL model in the plurality of DRL models, generating a second plurality of trading policies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of individual securities in the set of securities, and (iii) macroeconomic data; (C) determining a momentum score for each security in the set of securities; (D) selecting a candidate subset of securities from the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold; (E) for each security in the candidate subset of securities, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading policies to data associated with the security, (ii) generating a second set of one or moretrading signals for the security by applying the second plurality of trading policies to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security, wherein the aggregated trading signal indicates a signal strength; and (F) adding one or more securities from the candidate subset of securities to a trading portfolio based on the aggregated trading signals for the securities in the subset of securities.

[0359] Example 39: The computing system of Example 38, wherein the program instructions, when executed by the one or more processors, cause the computing system to perform functions comprising determining a portfolio weight for each security in the trading portfolio, wherein for an individual security in the trading portfolio, the portfolio weight for the individual security is proportional to a strength of the aggregated trading signal corresponding to the individual security, and wherein adding one or more securities from the candidate subset of securities to the trading portfolio based on the aggregated trading signals for the securities in the subset of securities comprises, for an individual security, at least one of: (A) adding the individual security to the trading portfolio as a long position when (i) the value of the aggregated trading signal for the individual security a positive value, (ii) the strength of the aggregated trading signal for the individual security is above a threshold strength, and (iii) a number of shares of the individual security added to the trading portfolio is based on the portfolio weight of the individual security; or (B) adding the individual security to the trading portfolio as a short position when (i) the value of the aggregated trading signal for the individual security is a negative value, (ii) the strength of the aggregated trading signal for the individual security is below a threshold strength, and (iii) a number of shares shorted in the trading portfolio is based on the portfolio weight of the individual security.

[0360] Example 40: The computing system of Example 38 or 39, wherein the program instructions, when executed by the one or more processors, cause the computing system to perform functions comprising monitoring changes in the aggregate trading signals for each security in the trading portfolio, and removing an individual security from the trading portfolio based on a change in the aggregate trading signal corresponding to the individual security.15. Conclusions

[0361] In accordance with various embodiments of the present disclosure, one or more aspects of the methods described herein are intended for operation as software programs and / or components thereof running on one or more computer processors. Furthermore, software implementations can include, but are not limited to, distributed processing or component / object distributed processing, parallel processing, and / or virtual machine processing, any one of which (or combination thereof) can also be used to implement the methods described herein.

[0362] The present disclosure contemplates a tangible, non-transitory machine-readable medium containing instructions so that interconnected computing devices connected via communications networks can exchange voice, video and / or other data with each other in connection with executing the disclosed processes. The instructions may further be transmitted or received over one or more communications networks between and among computing devices.

[0363] While the machine-readable media may be shown in example embodiments as a single medium, the term “machine-readable medium” or “machine-readable media” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” or “machine-readable media” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the present disclosure.

[0364] The terms “machine-readable medium,” “machine-readable device,” or “computer- readable device” shall accordingly be taken to include, but not be limited to: memory devices, solid-state memories such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories; magneto-optical or optical medium such as a disk or tape; or other self-contained information archive or set of archives is considered a distribution medium equivalent to a tangible storage medium. The “machine-readable medium,” “machine-readable device,” or “computer-readable device” may be non-transitory, and, in certain embodiments, may not include a wave or signal per se. Accordingly, the disclosure is considered to include any one or more of a machine- readable medium or a distribution medium, as listed herein and including art-recognized equivalents and successor media, in which the software implementations herein are stored.

[0365] The illustrations of arrangements described herein are intended to provide a general understanding of the structure of various embodiments, and they are not intended to serve as a complete description of all the elements and features of apparatus and systems that might make use of the structures described herein. Other arrangements may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Figures are also merely representational and may not be drawn to scale. Certain proportions thereof may be exaggerated, while others may be minimized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0366] Thus, although specific arrangements have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific arrangement shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments and arrangements of the invention. Combinations of the above arrangements, and other arrangements not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description. Therefore, it is intended that the disclosure not be limited to the particular arrangement s) disclosed as the best mode contemplated for carrying out this invention, but that the invention will include all embodiments and arrangements falling within the scope of the appended claims. The foregoing is provided for purposes of illustrating, explaining, and describing embodiments of this invention. Modifications and adaptations to these embodiments will be apparent to those skilled in the art and may be made without departing from the scope or spirit of this invention. Upon reviewing the aforementioned embodiments, it would be evident to an artisan with ordinary skill in the art that said embodiments can be modified, reduced, or enhanced without departing from the scope and spirit of the claims described below.

Claims

CLAIMSWhat is claimed is:

1. A method performed by a computing system, the method comprising: receiving an actual dataset comprising a time-ordered sequence of actual historical price data of an actual security during a timeframe, wherein the actual security is listed on a financial exchange, and wherein the time-ordered sequence of actual historical price data comprises a market price of the actual security on the financial exchange at each time point of a plurality of time points during the timeframe; based on the actual dataset, generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe, wherein each fictional security of the plurality of fictional securities is not listed on any financial exchange, wherein an individual synthetic dataset for an individual fictional security comprises a synthetic time-ordered sequence of historical price data for the fictional security, and wherein the time-ordered sequence of synthetic historical price data comprises a synthetic price of the fictional security at each time point of the plurality of time points during the timeframe; and training a machine learning model with data comprising the actual dataset and the plurality of synthetic datasets, wherein after training the machine learning model, the machine learning model is configured to forecast a future price data of the actual security.

2. The method of claim 1, wherein generating a plurality of synthetic datasets for a plurality of fictional securities during the timeframe comprises, for an individual fictional security: for each market price of the actual security at each time point of the plurality of time points during the timeframe, generating a synthetic price at the time point by multiplying the market price by a random number, wherein market prices at different time points are multiplied by different random numbers to generate the synthetic prices at the different time points; and ordering the generated synthetic prices based on their corresponding time points into a synthetic dataset for the individual fictional security.

3. The method of claim 2, wherein the random number is within a range defined by a lower bound and an upper bound.

4. The method of claim 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises: multiplying the market price by a random number that is less than one, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point.

5. The method of claim 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises: multiplying the market price by a random number that is greater than one, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point.

6. The method of claim 2, wherein generating a synthetic price at the time point by multiplying the market price by a random number comprises: generating a synthetic price at the time point by multiplying the market price by a first random number based on a second random number that is different than the first random number, wherein: when the second random number has a first attribute, the first random number is less than one, thereby resulting in a corresponding synthetic price that is lower than the market price at the time point; and when the second random number has a second attribute, the first random number is greater than one, thereby resulting in a corresponding synthetic price that is higher than the market price at the time point.

7. The method of claim 1, wherein training the machine learning model with data comprising the actual dataset and the plurality of synthetic datasets comprises: for the actual security, extracting one or more features from price data in the actual dataset; for the plurality of fictional securities, for each fictional security, extracting the one or more features from price data in the synthetic dataset for the fictional security; andtraining the model to predict the future price of the actual security based at least in part on the one or more extracted features from the price data in the actual dataset and the one or more extracted features from the price data in each synthetic dataset.

8. The method of claim 7, wherein the one or more features, comprise one or more of: (i) a Relative Strength Index (RSI) that quantifies a speed and change of price movement of the price data, (ii) a Moving Average Convergence / Divergence (MACD) using two exponential moving averages of different timeframes to identify a strength of a directional move in the price data, (iii) an Average Direction Index (ADX) that measures a strength and momentum of change in the price data, (iv) a Stochastic Oscillator that measures a current price relative to a price range over a number of periods, (v) a Simple Moving Average (SMA) that measures an average price over a specific time period, or (vi) a Standard Deviation that measures volatility in the price data.

9. The method of claim 1, wherein generating the plurality of synthetic datasets for the plurality of fictional securities during the timeframe comprises, for an individual synthetic dataset for an individual fictional security: applying a bi-stage transformation to each data point Si of the actual dataset to produce a corresponding synthetic data point pi for the synthetic dataset, wherein the bi-stage transformation is defined by an equation pi = (si0i) \| / iSi; wherein Oi represents a maximum deviation percentage of pi from Si, and <| / i represents an associativity factor that determines a direction of deviation that is positive or negative.

10. The method of claim 9, further comprising: selecting a value for 0i within a predetermined range of 0 to L, where L is a predefined maximum deviation limit, and Oi is a real number between 0 and L; and setting \| / i to a positive or negative value, wherein (i) when \| / i is positive, synthetic price pi is greater than or equal to actual price Si, and (ii) when \| / i is negative, synthetic price pi is less than or equal to actual price Si.11 . The method of claim 1 , further comprising: generating a set of securities comprising (i) a plurality of actual securities and (ii) a plurality of fictional securities for each actual security in the plurality of actual securities; for a first Deep Reinforcement Learning (DRL) model in a plurality of DRL models, generating a first plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data; for a second DRL model in the plurality of DRL models, generating a second plurality of trading strategies based at least in part on one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data; determining a momentum score for each actual security in the set of securities; selecting a candidate subset of securities from the actual securities in the set of securities, wherein individual securities in the candidate subset of securities have a momentum score that satisfies a momentum threshold; for each security in the candidate subset of securities, (i) generating a first set of one or more trading signals for the security by applying the first plurality of trading strategies to data associated with the security, (ii) generating a second set of one or more trading signals for the security by applying the second plurality of trading strategies to data associated with the security, and (iii) generating an aggregated trading signal for the security based on the first set of one or more trading signals for the security and the second set of one or more trading signals for the security, wherein the aggregated trading signal indicates a signal strength; and adding one or more securities from the candidate subset of securities to a trading portfolio based on the aggregated trading signals for the securities in the subset of securities.

12. The method of claim 11, wherein: generating the first plurality of trading strategies for the first DRL model comprises using a first DRL agent to learn the first plurality of trading strategies by using a first DRL algorithm to interact with an environment comprising the (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data;generating the second plurality of trading strategies for the second DRL model comprises using a second DRL agent to learn the second plurality of trading strategies by using a second DRL algorithm to interact with the environment comprising the (i) technical features corresponding to individual securities in a set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data; wherein the first DRL algorithm comprises one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm; and wherein the second DRL algorithm comprises one of (i) an Advantage Actor Critic (A2C) algorithm, (ii) a Deep Q-Networks (DQN) algorithm, or (iii) a Proximal Policy Optimization (PPO) algorithm.

13. The method of claim 11, wherein generating the first plurality of trading strategies for the first DRL model comprises using a first DRL agent to learn the first plurality of trading strategies by using a first DRL algorithm to interact with an environment comprising the (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data comprises: generating at least one DRL policy that maps a state to an action; wherein the state for the at least one DRL policy corresponds to a set of one or more (i) technical features corresponding to individual securities in the set of securities, (ii) fundamental features of the actual individual securities in the set of securities, and (iii) macroeconomic data; wherein the action for the at least one DRL policy corresponds to one of buying or selling a security; and wherein an individual trading strategy in the first plurality of trading strategies is based at least in part on the at least one DRL policy.

14. The method of claim 11, wherein determining a momentum score for each actual security in the set of securities comprises, for an individual actual security in the set of securities: calculating a fast momentum score for the individual actual security based on a one month return of the actual security divided by a volatility of the actual security;calculating a z-score for the fast momentum score by determining a difference between a mean of momentums of all actual securities in the set of securities and the fast momentum score for the individual actual security, and dividing the difference by a standard deviation of momentums of all actual securities in the set of securities; calculating a slow momentum score for the individual actual security based on a six month return of the security divided by the volatility of the actual security; calculating a z-score for the slow momentum score for the individual actual security by determining a difference between a mean of momentums of all actual securities in the set of securities and the slow momentum score for the individual actual security, and dividing the difference by the standard deviation of momentums of all actual securities in the set of securities; and determining the momentum score for the individual actual security by determining a sum of the z-score for the fast momentum for the individual actual security and the z-score of the slow momentum for the individual actual security, and dividing the sum by two to produce the momentum score for the individual actual security.

15. Tangible, non-transitory computer readable media comprising program instructions, wherein the program instructions, when executed by one or more processors, cause a computing system to perform the method of any of claim 1 through claim 14.

Images