Hierarchical probability modeling method and system based on residual information entropy and storage medium

Abstract

The application discloses a hierarchical probability modeling method and system based on residual information entropy and a storage medium, relates to the technical field of modeling, and has the technical scheme as follows: adaptive clustering is performed on pretreated input data to obtain a plurality of data subsets; a probability density model of a current layer is reconstructed based on a best probability distribution fitting, and a residual between the probability density model and a real probability density corresponding to the input data is calculated; a normalized information entropy of the residual is calculated, and it is judged whether the normalized information entropy is greater than a preset entropy threshold; if the normalized information entropy is less than or equal to the preset entropy threshold, the current residual is taken as new input data; if the normalized information entropy is greater than the preset entropy threshold, modeling is stopped, and a hierarchical probability model is output, so that model analysis and optimization are facilitated.

Description

Technical Field

[0001] This invention relates to the field of modeling technology, and more specifically, to a hierarchical probabilistic modeling method, system, and storage medium based on residual information entropy. Background Technology

[0002] Probability density modeling depicts the true distribution patterns of data, providing support for subsequent tasks such as probability inference and anomaly detection. Traditional probability modeling often uses a single preset distribution to fit the global data, such as directly using a normal distribution, which cannot adapt to the heterogeneous distribution characteristics of multimodal data.

[0003] In real-world scenarios, input data often consists of multiple subgroups with different distributions. A single distribution cannot simultaneously cover the distribution patterns of different subgroups, resulting in severely insufficient fitting accuracy.

[0004] Some improved methods use Gaussian mixture models, but the number of clusters needs to be preset manually, the parameter settings depend on human experience, which is highly subjective, and since they only use Gaussian distribution as a component, they cannot adapt to non-Gaussian subgroups, thus limiting the accuracy of modeling. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a hierarchical probabilistic modeling method, system and storage medium based on residual information entropy.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The hierarchical probabilistic modeling method based on residual information entropy includes the following steps: Adaptive clustering is performed on the preprocessed input data to obtain multiple data subsets; For each subset of data, calculate the first to fourth order moment statistics, and select the best probability distribution from the preset probability distribution family for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching. The probability density model of the current layer is reconstructed based on the best fitted probability distribution, and the residual between the probability density model and the true probability density corresponding to the input data is calculated. Calculate the normalized information entropy of the residual and determine whether the normalized information entropy is greater than the preset entropy threshold. If the normalized information entropy is less than or equal to the preset entropy threshold, then the current residual will be used as the new input data. If the normalized information entropy is greater than the preset entropy threshold, modeling stops and a hierarchical probability model is output.

[0007] Preferably, adaptive clustering uses a Bayesian-Gaussian mixture model to automatically determine the number of clusters.

[0008] Preferably, the preset probability distribution family includes the normal distribution, Gamma distribution, t distribution, and log-normal distribution.

[0009] Preferably, calculating the normalized information entropy of the residual specifically includes the following steps: Histogram statistics are performed on the residuals to obtain their probability distribution. The information entropy of the probability distribution is calculated, and the information entropy is normalized after being divided by the maximum possible entropy value.

[0010] Preferably, the probability density model of the current layer is reconstructed based on the best-fit probability distribution, specifically including the following steps: Extract the feature parameters of the optimal probability distribution corresponding to the data subset, and obtain the clustering characteristics through adaptive clustering based on the feature parameters; The degree of clustering density is determined based on the skewness and kurtosis characteristics of the moment statistics. The probability density structure of the current layer is obtained by fusing the optimal probability distribution based on the aggregation characteristics and the degree of aggregation.

[0011] Preferably, the residual between the probability density model and the true probability density corresponding to the input data is calculated, specifically including the following steps: Adaptive calibration of the probability density model; Extract the actual distribution features of each data point in the input data, and characterize the true probability density of the input data by combining the distribution differences of the data subsets; The corrected probability density model is compared point by point with the true probability density obtained by characterization to obtain the distribution deviation. The distribution deviation of each data point is assigned a corresponding weight based on the fitting accuracy of the data subset; Based on the weights, the distribution deviations of all data points in the input data are integrated to obtain the residual between the probability density model and the true probability density corresponding to the input data.

[0012] Preferably, the optimal probability distribution is selected from a preset family of probability distributions for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching, specifically including the following steps: Based on the skewness and kurtosis characteristics of the moment statistics, feature extraction is performed on each candidate distribution of the preset probability distribution family to obtain the asymmetry and steepness characteristics of each candidate distribution. Candidate distributions whose asymmetry characteristics deviate from the statistic within a preset range are used as the initial candidate set. The steepness statistic of the data subset is aligned and compared with the steepness features of each candidate distribution in the candidate set to obtain the degree of bias. The candidate distributions are then ranked according to the degree of bias, and the candidate distribution with the smallest degree of bias is selected as the distribution to be verified. The optimal probability distribution is obtained by performing a fit test on the distribution to be verified, and the data subset is fitted based on the optimal probability distribution.

[0013] Preferably, the optimal probability distribution is obtained by performing a fit check on the distribution to be verified, specifically including the following steps: Verify the fit between the distribution to be verified and the distribution characteristics of the corresponding subset of data; If the fit meets the preset standard, then the distribution to be verified is determined to be the optimal probability distribution.

[0014] A hierarchical probabilistic modeling system based on residual information entropy includes: The clustering analysis module performs adaptive clustering on the preprocessed input data to obtain multiple data subsets; The moment matching distribution fitting module calculates first to fourth-order moment statistics for each data subset, and selects the best probability distribution from a preset probability distribution family for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching. The residual calculation module reconstructs the probability density model of the current layer based on the best-fit probability distribution and calculates the residual between the probability density model and the true probability density corresponding to the input data. The information entropy analysis module calculates the normalized information entropy of the residuals and determines whether the normalized information entropy is greater than a preset entropy threshold. If the normalized information entropy is less than or equal to the preset entropy threshold, the decision control module will use the current residual as the new input data. If the normalized information entropy is greater than the preset entropy threshold, the model output module stops modeling and outputs a hierarchical probability model.

[0015] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a hierarchical probabilistic modeling method based on residual information entropy.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention adaptively clusters the preprocessed input data, splitting the overall data into multiple subsets with consistent characteristics. For each subset, first to fourth-order moment statistics are calculated. Based on the skewness and kurtosis characteristics of the moment statistics, the optimal probability distribution is selected from a pre-defined probability distribution family for fitting using moment matching. This allows for matching the optimal model to the distribution characteristics of different subsets, constructing a hierarchical probabilistic modeling system that enhances the model's generalization and expressive capabilities. After reconstructing the current layer's probability density model based on the best-fit distribution, the residual between the model and the true probability density is calculated. The normalized information entropy of the residual is used as the basis for iterative judgment, forming a hierarchical modeling iteration logic. When the normalized information entropy is less than or equal to a preset threshold, the residual is used as new input data for the next round of modeling, continuously optimizing the regions where the model is not properly fitted. When the normalized information entropy is greater than the preset threshold, modeling stops and the final hierarchical probabilistic model is output. This approach can progressively capture subtle distributional features and complex patterns within the data, enabling the model to not only cover the main distribution of the data but also take into account local biases and detailed features. This significantly improves the model's generalization ability, making its modeling performance on complex multimodal data far superior to traditional single-layer probabilistic models. It also enhances the interpretability of the probabilistic model, facilitating model analysis and optimization. The hierarchical modeling structure of this approach is clear; each layer's probability density model corresponds to a subset of data and its distribution characteristics. By decomposing the model layer by layer, the distribution type and fitting parameters of each subset are clearly defined, effectively addressing the poor adaptability of traditional probabilistic modeling to complex data scenarios. Attached Figure Description

[0017] Figure 1 A schematic diagram illustrating the steps of a hierarchical probabilistic modeling method based on residual information entropy is provided for embodiments of the present invention. Figure 2 A schematic diagram of a hierarchical probabilistic modeling system based on residual information entropy is provided for embodiments of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0021] Reference Figures 1-2 As shown.

[0022] The embodiments further illustrate the hierarchical probabilistic modeling method, system, and storage medium based on residual information entropy proposed in this invention.

[0023] The hierarchical probabilistic modeling method based on residual information entropy includes the following steps: The input data is cleaned and standardized, and the preprocessed input data is adaptively clustered to obtain multiple data subsets; Adaptive clustering uses a Bayesian-Gaussian mixture model to automatically determine the number of clusters.

[0024] The preprocessed input data is standardized data that has undergone operations such as standardization, outlier removal, and missing value completion. This type of data ensures the reliability and stability of the clustering results. The core of adaptive clustering is the Bayesian Gaussian Mixture Model. The key characteristic that distinguishes the Bayesian Gaussian Mixture Model from the traditional Gaussian Mixture Model is that it does not require manual pre-setting of the number of clusters. It automatically determines the optimal number of clusters through the Bayesian inference mechanism. Based on the introduction of the Dirichlet process prior distribution, it automatically removes redundant components with extremely low weights by applying sparse priors to the component weights of the mixture model, and finally retains the true clustering structure that exists in the data distribution.

[0025] Bayesian Gaussian mixture models revolve around posterior inference of parameters. The model assumes that the input data follows a weighted mixture of multiple Gaussian distributions, and its probability density function can be expressed as: ,in The mixing weights for the k-th Gaussian component satisfy the following condition: N(x|μ k Let Σ(k) be the probability density function of the k-th Gaussian distribution. Let Σ be the mean vector of the corresponding components. k Let Σ be the covariance matrix. Within the Bayesian framework, each parameter is assigned a conjugate prior distribution, where the mixed weights π follow a Dirichlet prior distribution, the mean μ follows a normal prior distribution, and the covariance matrix Σ follows a Wissaud inverse prior distribution. The posterior distributions of each parameter are iteratively updated using Markov chain Monte Carlo sampling or variational inference methods. Simultaneously, leveraging the truncation property of the Dirichlet process, components with weights approaching 0 are automatically merged, ultimately determining the actual number of clusters k in the data. This adaptive selection of the number of clusters can be achieved without manual intervention.

[0026] After parameter inference, based on the maximum a posteriori probability criterion, a corresponding cluster label is assigned to each input data point, dividing the preprocessed input data into multiple data subsets. Each data subset corresponds to a homogeneous Gaussian distribution cluster, meaning that the data points within the subset have similar statistical distribution characteristics, meeting the input requirements for subsequent moment-matching distribution fitting. If the input data consists of multi-dimensional operating parameters collected by an industrial sensor, and the preprocessed data includes normal operating conditions, slightly abnormal operating conditions, and severely abnormal operating conditions, traditional Gaussian mixture models require manual pre-setting of three cluster numbers. If the pre-set number does not match the actual distribution, clustering will fail. However, the Bayesian Gaussian mixture model automatically identifies three real cluster structures, dividing the data into three corresponding data subsets. Each subset corresponds to parameter data for one type of operating condition, providing an accurate data foundation for subsequent probability distribution fitting for different operating conditions.

[0027] For each subset of data, calculate the first to fourth order moment statistics. Based on the skewness and kurtosis characteristics of the moment statistics, select the best probability distribution from a preset family of probability distributions for fitting using moment matching. The specific steps include: Based on the skewness and kurtosis characteristics of the moment statistics, feature extraction is performed on each candidate distribution of the preset probability distribution family to obtain the asymmetry and steepness characteristics of each candidate distribution. Candidate distributions whose asymmetry characteristics deviate from the statistic within a preset range are used as the initial candidate set. The steepness statistic of the data subset is aligned and compared with the steepness features of each candidate distribution in the candidate set to obtain the degree of bias. The candidate distributions are then ranked according to the degree of bias, and the candidate distribution with the smallest degree of bias is selected as the distribution to be verified. To obtain the optimal probability distribution by performing a fit check on the distribution to be verified, the specific steps include: Verify the fit between the distribution to be verified and the distribution characteristics of the corresponding subset of data; If the fit reaches the preset standard, then the distribution to be verified is determined to be the optimal probability distribution; Fit the data subset based on the optimal probability distribution; The preset probability distribution family includes the normal distribution, Gamma distribution, t distribution, and log-normal distribution.

[0028] For each subset of data obtained through adaptive clustering, the first to fourth moment statistics are calculated. The first moment represents the mean, the second moment the variance, the third moment the skewness, and the fourth moment the kurtosis. Skewness characterizes the asymmetry of the data distribution, and its calculation formula is as follows: Where E is the expectation operator, μ is the mean of the data subset, and σ is the standard deviation of the data subset. The distribution is perfectly symmetrical when skewness is 0, right-skewed when skewness is greater than 0, and left-skewed when skewness is less than 0. Kurtosis is used to characterize the steepness of the data distribution, and its calculation formula is... Typically, the kurtosis of a normal distribution (3) is used as a benchmark. A kurtosis greater than 3 indicates a steeper distribution with thicker tails than a normal distribution, while a kurtosis less than 3 indicates a flatter distribution with thinner tails. Asymmetry characteristics and kurtosis statistics corresponding to subsets of data are extracted based on skewness and kurtosis.

[0029] For each candidate distribution within a predefined family of probability distributions, their corresponding asymmetry and kurtosis features are extracted. The normal distribution has a theoretical skewness of 0 and a theoretical kurtosis of 3, classifying it as a perfectly symmetric distribution with moderate kurtosis; the Gamma distribution has a theoretical skewness of... The theoretical kurtosis is ,in Let be the shape parameter of the Gamma distribution, with a constant positive skewness, belonging to a right-skewed distribution, and kurtosis varying with the shape parameter; the theoretical skewness of the t distribution is 0, and the theoretical kurtosis is . ,in The t-distribution has [number] degrees of freedom, is a symmetrical distribution, has a kurtosis always greater than 3, and a thicker tail than the normal distribution; the theoretical skewness of the log-normal distribution is [value]. The theoretical kurtosis is , where σ is the log-normal standard deviation, the skewness is always positive, it belongs to the right-skewed distribution, and the kurtosis is much greater than that of the normal distribution.

[0030] Candidate distributions whose asymmetry characteristics and deviations from the skewness statistics of a data subset are within a preset range are selected as a preliminary candidate set. If the skewness statistic of a data subset is 0.1 and the preset deviation range is ±0.2, then the normal distribution and t distribution with a theoretical skewness value of 0 can enter the candidate set, while the Gamma distribution and log-normal distribution with a theoretical skewness value of positive are eliminated because the deviation exceeds the range. If the skewness statistic of a data subset is 1.2 and the preset deviation range is ±0.3, then the Gamma distribution and log-normal distribution enter the candidate set, while the normal distribution and t distribution are eliminated.

[0031] For each candidate distribution within the candidate set, the steepness statistic of the data subset is aligned and compared with the steepness characteristics of each candidate distribution one by one, and the degree of deviation between the two is calculated. The degree of deviation is calculated using the absolute deviation formula. ,,in The actual kurtosis of the data subset. Let be the theoretical kurtosis of the candidate distribution. The smaller the kurtosis, the higher the match between the candidate distribution and the data subset. The candidate distributions within the candidate set are sorted in ascending order based on their kurtosis, and the candidate distribution with the smallest kurtosis is selected as the distribution to be verified. If the actual kurtosis of a data subset is 4.2, and the candidate set includes a t-distribution and a normal distribution, where the theoretical kurtosis of the t-distribution is 4.1 and the theoretical kurtosis of the normal distribution is 3, then the kurtosis of the t-distribution is 0.1, and the kurtosis of the normal distribution is 1.2. Therefore, the t-distribution is selected as the distribution to be verified.

[0032] Verify the goodness of fit between the distribution to be verified and the corresponding distribution characteristics of the data subset. Various quantitative methods can be used to verify the goodness of fit, such as the chi-square goodness-of-fit test, the formula of which is... ,in This represents the actual observation frequency of the data subset in each interval. To determine the theoretical expected frequency of the distribution to be validated within the corresponding interval, the chi-square statistic is calculated and compared with a critical value to assess the goodness of fit. Alternatively, the Kolmogorov-Smirnov test can be used to evaluate the goodness of fit by calculating the maximum absolute deviation between the empirical and theoretical distribution functions. If the goodness of fit meets a preset standard, such as a p-value greater than 0.05 in the chi-square test or a statistic less than a preset threshold in the KS test, the distribution to be validated is determined to be the optimal probability distribution. If the goodness of fit does not meet the preset standard, the candidate distribution with the second smallest deviation in the candidate set can be selected as the new distribution to be validated, and the fit verification process can be repeated until the optimal probability distribution that meets the requirements is found.

[0033] The corresponding subset of data is fitted based on the determined optimal probability distribution, and the characteristic parameters of the distribution are extracted, such as the mean and variance of the normal distribution, the shape and scale parameters of the Gamma distribution, the degrees of freedom, location and scale parameters of the t distribution, and the log mean and log standard deviation of the log-normal distribution.

[0034] The probability density model of the current layer is reconstructed based on the best-fit probability distribution, and the residual between the probability density model and the true probability density corresponding to the input data is calculated. Specifically, the following steps are included: Adaptive calibration of the probability density model; Extract the actual distribution features of each data point in the input data, and characterize the true probability density of the input data by combining the distribution differences of the data subsets; The corrected probability density model is compared point by point with the true probability density obtained by characterization to obtain the distribution deviation. The distribution deviation of each data point is assigned a corresponding weight based on the fitting accuracy of the data subset; Based on the weights, the distribution deviation of all data points in the input data is integrated to obtain the residual between the probability density model and the true probability density corresponding to the input data. The probability density model of the current layer is a theoretical model obtained by fusing the best probability distributions of multiple data subsets. Its structure and parameters are generated based on clustering and moment matching, and there is a certain systematic deviation from the actual data distribution, thus requiring adaptive correction. Combining the clustering characteristics and density of the data subsets, the weights, positions, and scale parameters of each component distribution are dynamically adjusted. For components with lower fitting accuracy, the correction magnitude is appropriately increased; for components with higher fitting accuracy, parameter stability is maintained. Simultaneously, overlapping interference between components is eliminated, making the overall model more closely fit the global distribution characteristics of the input data. If a data subset corresponds to abnormal operating condition data from industrial sensors, and its fitted Gamma distribution has a tail deviation from the actual distribution, the correction process adjusts the shape and scale parameters of this distribution to reduce the tail gap between the theoretical distribution and the actual data. At the same time, the weight of this component in the global model is adjusted to avoid interference from abnormal components on the fitting of normal operating condition data, thus completing the adaptive optimization of the model.

[0035] The actual distribution characteristics of each data point in the input data are extracted, and the true probability density of the input data is characterized by combining the distribution differences of data subsets. This restores the true distribution of the input data and avoids the assumption bias of the theoretical model. First, for each data point, the local statistical characteristics of its data subset are extracted, including local mean, local variance, local skewness, and local kurtosis. The position and distribution contribution of this data point in the global data are also recorded. Combining the distribution differences between different data subsets, such as the distribution shift between normal and abnormal operating condition subsets, and the difference in the distribution span of different operating state subsets, the kernel density estimation method is used to characterize the global true probability density. The formula for kernel density estimation is as follows: Where n is the total number of input data samples, h is the bandwidth parameter, and K is the kernel function. For each input data point, no preset distribution pattern is required, and the true distribution pattern of the data can be restored. If the input data contains three data subsets: normal operation, minor fault, and severe fault, and the distribution center and span of the three subsets are significantly different, the representation process extracts the local distribution features of the three subsets respectively. Combining the distribution differences between subsets, a global true probability density covering the three operating conditions is generated through kernel density estimation, thus fully presenting the true distribution state of the input data.

[0036] The corrected probability density model is compared point-by-point with the represented true probability density to obtain the distribution bias. The point-by-point comparison covers the entire range of input data, calculating the theoretical and true probability density values ​​of the corrected model for each sampling point. The difference between the two values ​​quantifies the distribution bias. The formula for calculating the distribution bias is as follows: ,in To correct the model in The probability density value of a point. For the true probability density The probability density value of a point. This represents the distribution deviation for the corresponding data point. A larger deviation indicates a worse model fit at that point. For example, within the range of sensor data values, the density value of the corrected model and the true density value are calculated for each sampling point. The deviation is smaller in the normal operating condition area and relatively larger in the abnormal operating condition area. After comparing each point, a distribution deviation sequence covering the entire range is generated, providing basic data for subsequent weighted integration.

[0037] The distribution deviation of each data point is assigned a corresponding weight based on the fitting accuracy of the data subset. The core of this step is to achieve differentiated weighting of deviations, avoiding excessive interference from the deviations of subsets with lower fitting accuracy on the global residuals, while highlighting the deviation contribution of subsets with higher fitting accuracy. Fitting accuracy can be quantified using indicators such as the coefficient of determination or mean squared error. The higher the fitting accuracy of a subset, the smaller the deviation weight of its corresponding data points; conversely, the lower the fitting accuracy of a subset, the larger the deviation weight of its corresponding data points. The weight calculation formula can be set as follows: ,in The coefficient of determination for the subset containing the data points. The closer to 1, the higher the fitting accuracy, corresponding to the weight w. i The smaller the coefficient of determination, the greater the weight. For example, if the coefficient of determination for the normal operating condition data subset is 0.98 and the coefficient of determination for the abnormal operating condition data subset is 0.85, then the deviation weight for the normal operating condition data points is 0.02, and the deviation weight for the abnormal operating condition data points is 0.15. This achieves differentiated weighting of the deviations of subsets with different fitting accuracies, making the residuals more accurately reflect the true fitting defects of the model.

[0038] The distribution bias of all data points in the input data is integrated based on weights to obtain the residual between the probability density model and the corresponding true probability density of the input data. The integration process uses a weighted summation method, and the residual is calculated using the following formula: ,in Assigning weights to each data point. To determine the distribution deviation of the corresponding data points, a weighted average is used to eliminate the influence of dimensions, resulting in a globally unified residual sequence. This residual includes both the global deviation between the model and the true distribution and the fitting differences between different data subsets. It can be directly used as input data for the next layer of hierarchical modeling, driving iterative optimization of the model. The deviation of data points under normal and abnormal operating conditions, along with their corresponding weights, are substituted into the formula to calculate the global residual. If the normalized information entropy of the residual is less than a preset threshold, this residual is used as new input data to enter the next round of clustering, fitting, and residual calculation until the residual information entropy meets the stopping condition, ultimately completing the construction of the hierarchical probabilistic model. Through deviation quantification and weighted integration, accurate evaluation of the model fitting effect is achieved, providing a reliable driving signal for the iterative optimization of hierarchical modeling.

[0039] The calculation of the normalized information entropy of the residuals includes the following steps: Histogram statistics are performed on the residuals to obtain their probability distribution. The information entropy of the probability distribution is calculated, and the information entropy is normalized after being divided by the maximum possible entropy value.

[0040] The probability density model of the current layer is reconstructed based on the best-fitted probability distribution, specifically including the following steps: Extract the feature parameters of the optimal probability distribution corresponding to the data subset, and obtain the clustering characteristics through adaptive clustering based on the feature parameters; The degree of clustering density is determined based on the skewness and kurtosis characteristics of the moment statistics. The probability density structure of the current layer is obtained by fusing the optimal probability distribution based on the aggregation characteristics and the degree of aggregation.

[0041] Feature parameters of the optimal probability distribution corresponding to the data subsets are extracted, and clustering characteristics are obtained through adaptive clustering based on these parameters. Each data subset obtained through adaptive clustering is filtered for its optimal probability distribution using moment matching. For example, the normal distribution corresponds to the mean and variance, the Gamma distribution to the shape and scale parameters, the t distribution to the position and scale parameters, and the log-normal distribution to the logarithmic mean and logarithmic standard deviation. After extracting these feature parameters, the clustering characteristics obtained from Bayesian-Gaussian mixture model clustering—namely, the structural information such as the cluster center, cluster radius, and sample proportion of each data subset—are combined to determine the position and contribution weight of each distribution in the global model. If the input data consists of multi-condition operating parameters of an industrial sensor, clustering yields three data subsets: normal, slightly abnormal, and severely abnormal. The corresponding optimal distributions are the normal, Gamma, and log-normal distributions, respectively. After extracting the feature parameters of each distribution, the basic weights and positional benchmarks of the three distributions in the global model are determined by combining the sample proportions and center positions obtained from clustering.

[0042] The degree of clustering is determined based on the skewness and kurtosis characteristics of moment statistics. Skewness characterizes the asymmetry of the data distribution, while kurtosis characterizes the steepness of the data distribution; together, they reflect the degree of clustering within a data subset. A skewness closer to 0 indicates a more symmetrical distribution and more regular data clustering; a kurtosis closer to 3 indicates a steeper distribution, closer to a normal distribution, and more concentrated data clustering; a larger absolute value of skewness or a greater deviation of kurtosis from 3 indicates a more dispersed data distribution and lower clustering. By calculating the skewness and kurtosis of each data subset, the degree of clustering within the subset is quantitatively assessed, providing a basis for subsequent weight adjustments in distribution fusion. For example, a normal operating condition data subset with a skewness close to 0 and a kurtosis close to 3 indicates tight data clustering and high fitting accuracy; a severely abnormal operating condition data subset with a large absolute value of skewness and a kurtosis much greater than 3 indicates a dispersed data distribution and low clustering.

[0043] The probability density structure of the current layer is obtained by fusing the optimal probability distributions based on clustering characteristics and cluster density. The fusion process uses the clustering characteristics obtained from clustering as the basic framework and the cluster density as the basis for weight adjustment, weighting the optimal probability distributions of each data subset. Subsets with higher cluster density have larger fusion weights, and vice versa. Simultaneously, the position and scale parameters of the distributions are adjusted by considering the sample proportion and center position of each subset to eliminate overlapping interference between distributions, ultimately generating a unified probability density structure for the current layer. For example, the normal operating condition subset has a high cluster density, so the fusion weight is set to 0.7; the slightly abnormal subset has a weight of 0.2; and the severely abnormal subset has a weight of 0.1. The three distributions are then weighted and fused to obtain a current layer probability density model covering all operating conditions, ensuring the fitting accuracy for normal operating conditions while also taking into account the distribution characteristics of abnormal operating conditions.

[0044] Histogram statistics are performed on the residuals to obtain their probability distribution. The residual is the deviation sequence between the probability density model and the true probability density of the input data. The residual sequence is binned, dividing the range of residual values ​​into several equal-width intervals. The number of residual samples in each interval is counted, and the sample frequency of each interval is calculated to obtain the empirical probability distribution of the residuals.

[0045] Information entropy is used to quantify the uncertainty of probability distributions, and its calculation formula is as follows: ,in Let be the residual probability of the i-th interval. The larger the information entropy, the more dispersed the distribution of the residuals, the more information they contain, and the more room there is for model optimization. The smaller the information entropy, the more concentrated the distribution of the residuals, the less information they contain, and the more perfect the model fit.

[0046] Determine whether the normalized information entropy is greater than a preset entropy threshold; If the normalized information entropy is less than or equal to the preset entropy threshold, then the current residual will be used as the new input data. If the normalized information entropy is greater than the preset entropy threshold, modeling stops and a hierarchical probability model is output.

[0047] The information entropy is divided by the maximum possible entropy value and then normalized. The maximum possible entropy value is the information entropy when the residual distribution is perfectly uniform, and its calculation formula is: Where N is the number of bins in the histogram, with a perfectly uniform distribution having the highest uncertainty, corresponding to the maximum entropy value. The calculated information entropy H is divided by the maximum possible entropy value. The normalized information entropy is obtained. After normalization, the information entropy ranges from 0 to 1, where 0 represents perfectly concentrated residuals with no information content, and 1 represents perfectly uniform residuals with maximum information content. For example, if the number of residual bins is 10, the maximum possible entropy is... 10≈3.32, the actual calculated information entropy is 1.66, and the normalized information entropy is 0.5, indicating that the residual contains a moderate amount of information, which is used to determine whether to continue iterative modeling. If the normalized information entropy is less than or equal to the preset threshold, it means that the residual information is insufficient, and the residual is used as new input data to enter the next layer of modeling; if it is greater than the preset threshold, it means that the residual information is sufficient, modeling stops, and the final hierarchical probability model is output.

[0048] In the scenario of detecting abnormal financial transactions, bank transaction data is used as the processing object. The input data includes transaction records with features such as amount, time, location, and frequency. First, the data undergoes preprocessing operations such as standardization and missing value handling. Then, hierarchical modeling is performed using the modeling method of this invention. The modeling consists of three layers: the first layer uses adaptive clustering to discover three main normal transaction patterns, constructing a basic distribution model for normal transactions; the second layer constructs two sub-clusters for boundary transaction data, refining the distribution characteristics of boundary transactions; the third layer uses residual entropy as the criterion for continuous iterative modeling, stopping when the residual entropy reaches 0.85, thus completing the detection of abnormal patterns. This method identifies five normal transaction patterns and simultaneously detects three types of abnormal transaction patterns. Compared with traditional methods, it reduces the false alarm rate and can generate interpretable reports, achieving accurate identification and root cause tracing of anomalies.

[0049] In a medical image analysis scenario, targeting nodule detection in lung CT images, the input data consists of 1000 lung CT images. A CNN network is used to extract 200-dimensional feature vectors. These feature vectors are then used as input to the modeling method of this invention to construct a hierarchical probabilistic model in the feature space. The first layer identifies the probability distribution pattern of healthy tissues, establishing a baseline distribution for healthy tissues. The second layer distinguishes the distribution characteristics of different tissue types, achieving modeling of distribution differences among different tissues. The third layer performs residual iterative modeling for minute abnormal regions, completing the accurate detection of nodule abnormalities. This method offers higher nodule detection sensitivity, a lower false positive rate, and provides probabilistic diagnostic results, offering a reliable reference for clinical diagnosis.

[0050] In predictive maintenance scenarios for industrial equipment, using wind turbine fault prediction as an application, the input data includes time-series data of vibration, temperature, current, and voltage. Time-domain and frequency-domain features are extracted through feature engineering, and a hierarchical probabilistic model is constructed using the modeling method of this invention. Model construction consists of three stages: first, a hierarchical probabilistic model of the equipment's normal state is built to establish a baseline distribution for healthy equipment operation; second, real-time collected equipment data is compared point-by-point with the normal state model to calculate the distribution residuals; finally, fault early warning is achieved through residual monitoring. This method can predict equipment faults in advance with higher accuracy, effectively reducing unplanned downtime and lowering equipment maintenance costs, thus achieving early warning and preventative maintenance of equipment faults.

[0051] A hierarchical probabilistic modeling system based on residual information entropy includes: The clustering analysis module performs adaptive clustering on the preprocessed input data to obtain multiple data subsets; The moment matching distribution fitting module calculates first to fourth-order moment statistics for each data subset, and selects the best probability distribution from a preset probability distribution family for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching. The residual calculation module reconstructs the probability density model of the current layer based on the best-fit probability distribution and calculates the residual between the probability density model and the true probability density corresponding to the input data. The information entropy analysis module calculates the normalized information entropy of the residuals and determines whether the normalized information entropy is greater than a preset entropy threshold. If the normalized information entropy is less than or equal to the preset entropy threshold, the decision control module will use the current residual as the new input data. If the normalized information entropy is greater than the preset entropy threshold, the model output module stops modeling and outputs a hierarchical probability model.

[0052] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a hierarchical probabilistic modeling method based on residual information entropy.

[0053] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hierarchical probabilistic modeling method based on residual information entropy, characterized in that, Includes the following steps: Adaptive clustering is performed on the preprocessed input data to obtain multiple data subsets; For each subset of data, calculate the first to fourth order moment statistics, and select the best probability distribution from the preset probability distribution family for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching. The probability density model of the current layer is reconstructed based on the best fitted probability distribution, and the residual between the probability density model and the true probability density corresponding to the input data is calculated. Calculate the normalized information entropy of the residual and determine whether the normalized information entropy is greater than the preset entropy threshold. If the normalized information entropy is less than or equal to the preset entropy threshold, then the current residual will be used as the new input data. If the normalized information entropy is greater than the preset entropy threshold, modeling stops and a hierarchical probability model is output.

2. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, Adaptive clustering uses a Bayesian-Gaussian mixture model to automatically determine the number of clusters.

3. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, The preset probability distribution family includes the normal distribution, Gamma distribution, t distribution, and log-normal distribution.

4. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, The calculation of the normalized information entropy of the residuals includes the following steps: Histogram statistics are performed on the residuals to obtain their probability distribution. The information entropy of the probability distribution is calculated, and the information entropy is normalized after being divided by the maximum possible entropy value.

5. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, The probability density model of the current layer is reconstructed based on the best-fitted probability distribution, specifically including the following steps: Extract the feature parameters of the optimal probability distribution corresponding to the data subset, and obtain the clustering characteristics through adaptive clustering based on the feature parameters; The degree of clustering density is determined based on the skewness and kurtosis characteristics of the moment statistics. The probability density structure of the current layer is obtained by fusing the optimal probability distribution based on the aggregation characteristics and the degree of aggregation.

6. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, Furthermore, the residual between the probability density model and the true probability density corresponding to the input data is calculated, specifically including the following steps: Adaptive calibration of the probability density model; Extract the actual distribution features of each data point in the input data, and characterize the true probability density of the input data by combining the distribution differences of the data subsets; The corrected probability density model is compared point by point with the true probability density obtained by characterization to obtain the distribution deviation. The distribution deviation of each data point is assigned a corresponding weight based on the fitting accuracy of the data subset; Based on the weights, the distribution deviations of all data points in the input data are integrated to obtain the residual between the probability density model and the true probability density corresponding to the input data.

7. The hierarchical probabilistic modeling method based on residual information entropy according to claim 1, characterized in that, Based on the skewness and kurtosis characteristics of the moment statistics, the best probability distribution is selected from a preset family of probability distributions for fitting using moment matching. The specific steps include: Based on the skewness and kurtosis characteristics of the moment statistics, feature extraction is performed on each candidate distribution of the preset probability distribution family to obtain the asymmetry and steepness characteristics of each candidate distribution. Candidate distributions whose asymmetry characteristics deviate from the statistic within a preset range are used as the initial candidate set. The steepness statistic of the data subset is aligned and compared with the steepness features of each candidate distribution in the candidate set to obtain the degree of bias. The candidate distributions are then ranked according to the degree of bias, and the candidate distribution with the smallest degree of bias is selected as the distribution to be verified. The optimal probability distribution is obtained by performing a fit test on the distribution to be verified, and the data subset is fitted based on the optimal probability distribution.

8. The hierarchical probabilistic modeling method based on residual information entropy according to claim 7, characterized in that, To obtain the optimal probability distribution by performing a fit check on the distribution to be verified, the specific steps include: Verify the fit between the distribution to be verified and the distribution characteristics of the corresponding subset of data; If the fit meets the preset standard, then the distribution to be verified is determined to be the optimal probability distribution.

9. A hierarchical probabilistic modeling system based on residual information entropy, applied to the hierarchical probabilistic modeling method based on residual information entropy as described in any one of claims 1 to 8, characterized in that, include: The clustering analysis module performs adaptive clustering on the preprocessed input data to obtain multiple data subsets; The moment matching distribution fitting module calculates first to fourth-order moment statistics for each data subset, and selects the best probability distribution from a preset probability distribution family for fitting based on the skewness and kurtosis characteristics of the moment statistics through moment matching. The residual calculation module reconstructs the probability density model of the current layer based on the best-fit probability distribution and calculates the residual between the probability density model and the true probability density corresponding to the input data. The information entropy analysis module calculates the normalized information entropy of the residuals and determines whether the normalized information entropy is greater than a preset entropy threshold. If the normalized information entropy is less than or equal to the preset entropy threshold, the decision control module will use the current residual as the new input data. If the normalized information entropy is greater than the preset entropy threshold, the model output module stops modeling and outputs a hierarchical probability model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the hierarchical probabilistic modeling method based on residual information entropy as described in any one of claims 1-8.

Images