A stochastic probabilistic statistical model

Abstract

The application discloses a random probability statistical model, and relates to the technical field of data processing and mathematical modeling. The model is based on probability distribution and statistical law to model and infer uncertain systems. The model comprises sequentially connected feature processing modules, probability distribution modules, parameter inference modules and evaluation verification modules. Through the fusion architecture of "deep learning + probability model", the powerful nonlinear mapping capability of the neural network is used to fit complex conditional probability, so that the traditional probability model has the ability to process unstructured data such as images and texts. Meanwhile, the improved variational inference (VI) or efficient sampling (such as NUTS, HMC) is adopted, so that the computational cost under large-scale data is greatly reduced while the approximation accuracy of the posterior distribution is ensured, and the inference is efficient and accurate.

Description

Technical Field

[0001] This invention relates to the fields of data processing and mathematical modeling technology, specifically to a stochastic probability statistical model. Background Technology

[0002] The real world is rife with uncertainties, and traditional deterministic equations often fail to accurately describe the inherent patterns of these phenomena. Stochastic probability and statistical models, as a core tool for describing uncertainties using mathematical language, are a mathematical framework that uses "probability + statistical laws" to describe uncertain events, predict outcomes, and analyze data patterns. They characterize the uncertainty of data through probability distributions and utilize statistical inference to estimate unknown parameters or make predictions from observed data.

[0003] Its input or output is random, not a 100% deterministic functional relationship. Instead, it uses tools such as probability distribution, expectation, variance, likelihood, and Bayesian inference to characterize statistical patterns. Typical models include Naive Bayes, Hidden Markov Models (HMM), Gaussian Mixture Models (GMM), logistic regression, and Monte Carlo simulation models.

[0004] However, traditional stochastic probability and statistical models face many bottlenecks in practical industrial applications: (1) The distribution assumptions are too idealistic. They usually assume that the data follows a simple Gaussian distribution, which cannot fit the long-tailed, heavy-tailed, or multimodal characteristics that are common in the real world; (2) The structure is simple. Static models are difficult to deal with dynamic data with temporal and spatial correlations; (3) The inference computing power is a bottleneck. Traditional maximum likelihood estimation (MLE) or basic MCMC sampling is inefficient when facing high-dimensional massive data; (4) The expressive power is limited. It is difficult to capture the complex nonlinear features in the data; (5) There is a lack of uncertainty quantification. Traditional models have multiple output point estimates and cannot provide the confidence and risk boundary required for decision-making.

[0005] Therefore, there is an urgent need for a stochastic probability and statistics model that can comprehensively integrate distribution optimization, structural evolution, inference acceleration, and deep learning expression capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a stochastic probability statistical model that can comprehensively improve traditional models from dimensions such as probability distribution, model structure, parameter inference, feature processing, dynamic adaptation, computational efficiency, deep learning fusion, domain constraints, and evaluation system.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a stochastic probability statistical model, which models and infers uncertain systems based on probability distribution and statistical laws, and the model includes a sequentially connected feature processing module, a probability distribution module, a parameter inference module, and an evaluation and verification module; The model is improved through at least one of the following dimensions: Probability distribution improvement: Replace the basic probability distribution with a long-tailed distribution, heavy-tailed distribution, or time-varying probability distribution that fits the characteristics of the actual data; Model structure improvement: Introduce latent variables into the basic model to construct a hierarchical Bayesian structure, or introduce Markov transition structure to construct a temporal / spatial model; Improved parameter inference: Use variational inference VI or an improved Markov chain Monte Carlo (MCMC) algorithm to approximate the inference of the posterior distribution; Deep learning fusion improvement: Combine the neural network with the probability distribution module, and use the neural network to fit complex conditional probability distributions.

[0008] Preferably, in the improved probability distribution dimension, for observation data containing outliers, a t-distribution or a Gaussian mixture distribution (GMM) is used instead of the original Gaussian distribution to improve the model's robustness to long-tail characteristics and outliers in the data.

[0009] Preferably, in the feature processing improvement dimension, the feature processing module introduces sparse priors to the high-dimensional input data for feature selection and dimensionality reduction; for multi-source heterogeneous data, a probabilistic graphical model is used to fuse multi-source information.

[0010] Preferably, in the parameter inference improvement dimension, for missing data scenarios, the parameter inference module uses the expectation-maximization (EM) algorithm for data imputation and joint parameter estimation; when using variational inference (VI), the fitting accuracy of the approximate distribution is improved by optimizing a tighter variational lower bound.

[0011] Preferably, in the dynamic adaptive improvement dimension, the model is configured with a time-varying parameter module, which, by introducing a forgetting factor or adopting an incremental learning strategy, enables the model probability parameters to be dynamically updated over time without retraining all historical data.

[0012] Preferably, in the deep learning fusion improvement dimension, the model is specifically a deep probabilistic model, employing a variational autoencoder (VAE) or a Bayesian neural network (BNN) architecture; wherein, the hidden layer output of the neural network serves as the input parameter of the probability distribution module, realizing the fusion of nonlinear deep feature extraction and traditional probability statistics.

[0013] Preferably, in the evaluation and verification improvement dimension, the evaluation and verification module not only outputs the point prediction results, but also outputs the prediction variance and confidence interval calculated based on the posterior distribution, so as to realize the quantification of model uncertainty; the evaluation index adopts the log-likelihood value, calibration curve or KS test statistic.

[0014] This invention also provides a method for constructing a random probability statistical model, characterized by comprising the following steps: S1: Problem Analysis and Introduction of Domain Constraints: Based on the physical meaning of the actual application scenario, determine the input and output space of the model, and transform domain knowledge into model constraints or prior distributions; S2: Feature Engineering and Input Processing: Perform feature transformation, dimensionality reduction, or missing value imputation on the original data to make it meet the preset probability distribution assumptions; S3: Model Structure and Distribution Design: Select the corresponding probability distribution family based on the data characteristics, and configure a hierarchical structure or deep learning fusion structure; S4: Improved parameter inference: Set the loss function or likelihood function, and use improved variational inference or efficient MCMC sampling algorithm to solve the posterior distribution of model parameters; S5: Robustness and Dynamic Evaluation: Inject adversarial perturbations or outliers into the test set to verify the model's robustness, and verify the model's incremental update capability in an online environment. S6: Uncertainty Assessment and Deployment: Output the prediction results and confidence intervals, and deploy the model after verifying compliance based on comprehensive evaluation indicators.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention utilizes a fusion architecture of "deep learning + probabilistic model" to fit complex conditional probabilities using the powerful nonlinear mapping capabilities of neural networks, enabling traditional probabilistic models to process unstructured data such as images and text. At the same time, it employs improved variational inference (VI) or efficient sampling (such as NUTS and HMC) to significantly reduce computational overhead under large-scale data while ensuring the approximate accuracy of the posterior distribution, resulting in efficient and accurate inference. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the method in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 Please see Figure 1This invention provides a random probability statistical model, which models and infers uncertain systems based on probability distribution and statistical laws. The model includes a feature processing module, a probability distribution module, a parameter inference module, and an evaluation and verification module connected in sequence. The model is improved through at least one of the following dimensions: Feature processing improvements: The feature processing module introduces sparse priors for feature selection and dimensionality reduction of high-dimensional input data; for multi-source heterogeneous data, a probabilistic graphical model is used for multi-source information fusion. Probability distribution improvement: Replace the basic probability distribution with a long-tailed distribution, heavy-tailed distribution, or time-varying probability distribution that fits the characteristics of the actual data; In the improved probability distribution dimension, for observation data containing outliers, the t-distribution or Gaussian mixture distribution (GMM) is used instead of the original Gaussian distribution to improve the model's robustness to long-tailed data and outliers. Furthermore, by introducing heavy-tailed distributions, mixture distributions, and time-varying distributions, the stringent assumptions of traditional models regarding data distribution are overcome, significantly improving the model's ability to fit long-tailed and outlier data.

[0019] (3) Model structure improvement: Introduce latent variables into the basic model to construct a hierarchical Bayesian structure, or introduce Markov transition structure to construct a temporal / spatial model; (4) Improved parameter inference: Use variational inference VI or improved Markov chain Monte Carlo MCMC algorithm to approximate the inference of the posterior distribution; In the parameter inference improvement dimension, for missing data scenarios, the parameter inference module uses the expectation-maximization (EM) algorithm for data imputation and joint parameter estimation; when using variational inference (VI), the fitting accuracy of the approximate distribution is improved by optimizing a tighter variational lower bound.

[0020] (5) Evaluation and verification improvement: The evaluation and verification module not only outputs the point prediction results, but also outputs the prediction variance and confidence interval calculated based on the posterior distribution, so as to realize the quantification of model uncertainty; the evaluation index adopts the log-likelihood value, calibration curve or KS test statistic.

[0021] (6) Deep learning fusion improvement: Combine the neural network with the probability distribution module and use the neural network to fit the complex conditional probability distribution.

[0022] In the aforementioned deep learning fusion improvement dimension, the model is specifically a deep probabilistic model, employing a variational autoencoder (VAE) or a Bayesian neural network (BNN) architecture; wherein, the hidden layer output of the neural network serves as the input parameter of the probability distribution module, realizing the fusion of nonlinear deep feature extraction and traditional probability statistics.

[0023] (7) Dynamic adaptive improvement: The model is equipped with a time-varying parameter module. By introducing a forgetting factor or adopting an incremental learning strategy, the model probability parameters can be dynamically updated over time without retraining all historical data.

[0024] This invention provides an improved stochastic probability and statistical model, which is based on probability distribution and statistical laws and is constructed through at least one dimension, such as probability distribution improvement, model structure improvement, parameter inference improvement and deep learning fusion improvement.

[0025] Example 2 This embodiment provides a method for constructing a random probability statistical model, including the following steps: S1: Problem Analysis and Introduction of Domain Constraints: Based on the physical meaning of the actual application scenario, determine the input and output space of the model, and transform domain knowledge into model constraints or prior distributions; S2: Feature Engineering and Input Processing: Perform feature transformation, dimensionality reduction, or missing value imputation on the original data to make it meet the preset probability distribution assumptions; S3: Model Structure and Distribution Design: Select the corresponding probability distribution family based on the data characteristics, and configure a hierarchical structure or deep learning fusion structure; S4: Improved parameter inference: Set the loss function or likelihood function, and use improved variational inference or efficient MCMC sampling algorithm to solve the posterior distribution of model parameters; S5: Robustness and Dynamic Evaluation: Inject adversarial perturbations or outliers into the test set to verify the model's robustness, and verify the model's incremental update capability in an online environment. S6: Uncertainty Assessment and Deployment: Output the prediction results and confidence intervals, and deploy the model after verifying compliance based on comprehensive evaluation indicators.

[0026] This invention provides a method for constructing the above-mentioned model, which includes six steps: problem analysis and introduction of domain constraints, feature engineering, structure and distribution design, improved parameter inference, robustness assessment and uncertainty evaluation deployment.

[0027] Example 3 I. Financial Anti-Fraud Scenarios (Deep Learning Fusion + Probability Distribution Improvement) In the context of credit card transaction fraud prevention, fraudulent transactions account for an extremely low percentage (usually <0.5%), exhibiting an extreme long-tail / imbalanced distribution, and there are complex non-linear relationships between transaction characteristics.

[0028] This application's model strategy: Deep learning integration and improvement: Constructing a deep probabilistic model based on Bayesian neural networks (BNNs). The front end uses a multilayer perceptron (MLP) to extract deep nonlinear features of transactions; Improved probability distribution: Instead of using the traditional Sigmoid function to output deterministic probabilities, the output layer outputs the parameters (α,β) of the Beta distribution, transforming the classification problem into a probability distribution regression problem; Feature processing improvements: Dimensionality reduction is achieved by using embedding layers for high-dimensional category features to reduce noise interference; Evaluation and verification improvements: In addition to looking at the AUC index, a "calibration degree" index is introduced to assess the accuracy of the predicted probability and output the prediction confidence interval.

[0029] Table 1: Comparison of Improvement Dimensions between the Improved Model of this Invention and the Traditional Model

[0030] Table 2: Performance Comparison of Depth Probability Model under Different Distribution Assumptions in Example 1

[0031] The results show that by improving the probability distribution to a Beta distribution and introducing a heavy-tailed prior, the model not only improves its AUC and significantly enhances its calibration, but also quantifies the uncertainty of "how suspicious this transaction is" with extremely high accuracy. Its model output not only includes the predicted mean but also forces the output of the predicted variance and confidence interval, achieving comprehensive model uncertainty quantification and providing a basis for high-risk decision-making scenarios.

[0032] II. Predictive Maintenance of Industrial Equipment (Dynamic Adaptation + Model Structure Improvement) In industrial sensor monitoring scenarios, as equipment operates for longer periods, the statistical patterns of its vibration, temperature, and other data tend to drift (become non-stationary data). Traditional static models require periodic offline retraining, which is time-consuming and exhibits latency.

[0033] This application's model strategy: Model structure improvement: A dynamic probabilistic model with a hidden Markov transition structure is adopted to model the hidden states of the equipment (normal, slight wear, severe wear) as Markov chains; Dynamic adaptive improvement: Introducing a time-varying parameter mechanism. For newly arriving sensor data streams, incremental variational inference is used to update the transition probability matrix online, without reloading all historical data; Robustness improvement: A Gaussian noise model is added to the observation equation to describe the measurement error of the sensor itself, thereby improving the resistance to data contamination; Improved computational efficiency: Stochastic variational inference (SVI) is adopted, which only extracts a small batch of data for gradient updates each time, meeting the real-time requirements of industry.

[0034] Table 3: Efficiency and Accuracy of the Dynamic Adaptive Model in the Online Update Phase of Example 1

[0035] The results show that when data drift occurs, the performance of traditional models drops precipitously, while the present invention maintains a stable F1 score through dynamic adaptive improvement, and because it adopts incremental learning, memory usage does not increase, and a single update is completed in milliseconds.

[0036] In summary, the random probability statistical model provided by this invention achieves the following effects: 1. In view of the shortcomings of traditional models that "the distribution assumptions are too idealized", this invention has the beneficial effect of improving the fitting degree of complex data.

[0037] Traditional models often rigidly assume that data follows a simple Gaussian distribution, which is prone to distortion when faced with long-tailed, heavy-tailed, or multimodal data that are prevalent in the real world. This invention improves the dimensionality by introducing t-distributions, Beta distributions, Gaussian mixtures, and time-varying probability distributions to replace the simple basic distribution. This fundamentally relaxes the data assumptions of the model, enabling it to closely match the real physical laws of long-tailed and extremely imbalanced data, and significantly improves the accuracy of probability fitting.

[0038] 2. In view of the shortcomings of traditional models, such as "simple structure and lack of spatiotemporal correlation", this invention has the beneficial effect of enhancing the dynamic expression ability of complex systems.

[0039] Traditional static probabilistic models struggle to depict the evolution of data over time or spatial transformations. This invention improves the dimensionality of the model structure by introducing latent variables to construct a hierarchical Bayesian structure or embedding Markov transition mechanisms to build a temporal / spatial probabilistic model. This breaks through the limitations of the original single-layer, memoryless structure and endows the model with the ability to capture deep state transitions and complex spatiotemporal dependencies in the data.

[0040] 3. In view of the shortcomings of traditional models that have "bottleneck in inference computing power", this invention has the beneficial effect of realizing efficient inference of high-dimensional massive data.

[0041] Traditional maximum likelihood estimation (MLE) performs poorly with missing data, while basic MCMC sampling is extremely slow in high-dimensional parameter spaces and cannot adapt to big data scenarios. This invention improves dimensionality by enhancing parameter inference and computational efficiency. It employs an improved variational inference (VI) to optimize tighter lower bounds, or introduces efficient samplers such as NUTS and HMC, as well as distributed computing strategies. Without sacrificing inference accuracy, it reduces the computation time of complex integration and sampling from hours to milliseconds or seconds, breaking through the computational bottleneck.

[0042] 4. In view of the shortcomings of traditional models in terms of "limited expressive power", this invention has the beneficial effect of breaking through the bottleneck of nonlinear feature extraction.

[0043] Traditional probabilistic models (such as Naive Bayes and shallow logistic regression) rely on manual feature engineering, making it difficult to uncover complex nonlinear relationships in unstructured data. This invention improves dimensionality through deep learning integration, constructing a deep probabilistic architecture (such as VAE and BNN) that combines neural networks with probabilistic models. It leverages the powerful nonlinear mapping capabilities of neural networks to automatically extract deep features and use them as parameters for probability distributions, enabling traditional statistical models to possess top-level expressive capabilities for handling complex data such as images and text.

[0044] 5. In view of the shortcomings of traditional models in "lacking uncertainty quantification", this invention has the beneficial effect of providing interpretable and reliable decision boundaries.

[0045] Traditional models typically output definitive point estimates (such as single classification labels or regression values), failing to indicate the reliability of the predictions and easily leading to misjudgments in high-risk scenarios such as healthcare and finance. This invention improves the dimensionality through evaluation verification and uncertainty quantification, forcing the model to output prediction variance and confidence intervals based on the posterior distribution, and combining indicators such as log-likelihood and calibration for comprehensive evaluation. This effect enables the model not only to provide "what the prediction is," but also to clearly indicate "how confident the prediction is," providing crucial mathematical basis for downstream risk control.

[0046] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A stochastic probabilistic statistical model characterized in that, The model is based on probability distribution and statistical laws to model and infer uncertain systems. The model includes a sequentially connected feature processing module, probability distribution module, parameter inference module and evaluation and verification module. The model is improved through at least one of the following dimensions: Probability distribution improvement: Replace the basic probability distribution with a long-tailed distribution, heavy-tailed distribution, or time-varying probability distribution that fits the characteristics of the actual data; Model structure improvement: Introduce latent variables into the basic model to construct a hierarchical Bayesian structure, or introduce Markov transition structure to construct a temporal / spatial model; Improved parameter inference: Use variational inference VI or an improved Markov chain Monte Carlo (MCMC) algorithm to approximate the inference of the posterior distribution; Deep learning fusion improvement: Combine the neural network with the probability distribution module, and use the neural network to fit complex conditional probability distributions.

2. The random probability statistical model according to claim 1, characterized in that, In the improved probability distribution dimension, for observation data containing outliers, the original Gaussian distribution is replaced by a t-distribution or a Gaussian mixture distribution (GMM) to improve the model's robustness to long-tail characteristics and outliers.

3. The random probability statistical model according to claim 1, characterized in that, In the feature processing improvement dimension, the feature processing module introduces sparse priors to filter features and reduce dimensionality for high-dimensional input data; for multi-source heterogeneous data, a probabilistic graphical model is used to fuse multi-source information.

4. The random probability statistical model according to claim 1, characterized in that, In the parameter inference improvement dimension, for missing data scenarios, the parameter inference module uses the expectation-maximization (EM) algorithm for data imputation and joint parameter estimation; when using variational inference (VI), the fitting accuracy of the approximate distribution is improved by optimizing a tighter variational lower bound.

5. The random probability statistical model according to claim 1, characterized in that, In the dynamic adaptive improvement dimension, the model is configured with a time-varying parameter module. By introducing a forgetting factor or adopting an incremental learning strategy, the model probability parameters can be dynamically updated over time without retraining on all historical data.

6. The random probability statistical model according to claim 1, characterized in that, In the aforementioned deep learning fusion improvement dimension, the model is specifically a deep probabilistic model, employing a variational autoencoder (VAE) or a Bayesian neural network (BNN) architecture; wherein, the hidden layer output of the neural network serves as the input parameter of the probability distribution module, realizing the fusion of nonlinear deep feature extraction and traditional probability statistics.

7. The random probability statistical model according to claim 1, characterized in that, In the evaluation and verification improvement dimension, the evaluation and verification module not only outputs the point prediction results, but also outputs the prediction variance and confidence interval calculated based on the posterior distribution, so as to realize the quantification of model uncertainty; the evaluation index adopts the log-likelihood value, calibration curve or KS test statistic.

8. The method for constructing a random probability statistical model according to any one of claims 1-7, characterized in that, Includes the following steps: S1: Problem Analysis and Introduction of Domain Constraints: Based on the physical meaning of the actual application scenario, determine the input and output space of the model, and transform domain knowledge into model constraints or prior distributions; S2: Feature Engineering and Input Processing: Perform feature transformation, dimensionality reduction, or missing value imputation on the original data to make it meet the preset probability distribution assumptions; S3: Model Structure and Distribution Design: Select the corresponding probability distribution family based on the data characteristics, and configure a hierarchical structure or deep learning fusion structure; S4: Improved parameter inference: Set the loss function or likelihood function, and use improved variational inference or efficient MCMC sampling algorithm to solve the posterior distribution of model parameters; S5: Robustness and Dynamic Evaluation: Inject adversarial perturbations or outliers into the test set to verify the model's robustness, and verify the model's incremental update capability in an online environment. S6: Uncertainty Assessment and Deployment: Output the prediction results and confidence intervals, and deploy the model after verifying compliance based on comprehensive evaluation indicators.

Images