Random forest modeling method and system for automatic recognition of zdr arcs

Abstract

The application provides a ZDR arc automatic recognition random forest modeling method and system, relates to the meteorological radar data processing technical field, and includes the following steps: acquiring dual-polarization radar observation data, extracting a differential reflectivity field and a reflectivity field, determining an arc candidate area based on a spatial position relationship and extracting multi-dimensional feature parameters, calculating a mutual information matrix and a partial correlation coefficient matrix to screen feature pairs and perform tensor product operation to generate high-order interaction features, constructing a training sample set, initializing a basic random forest, and based on feature conditional entropy, reconstructing a sampling probability distribution to complete decision tree growth to obtain a basic classifier, extracting a decision tree discrimination path sequence, converting it into a topology graph, encoding it into a path embedding vector to train a meta-random forest classifier, inputting to-be-recognized data into the basic classifier and the meta-random forest classifier, and outputting a recognition result. The accuracy and automation degree of ZDR arc recognition are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of meteorological radar data processing technology, and in particular to a random forest modeling method and system for automatic ZDR arc identification. Background Technology

[0002] This technical field relates to weather phenomenon identification using dual-polarization radar data, particularly the automatic detection of differential reflectivity arcs (ZDR arcs). Current conventional methods typically rely on threshold judgments based on radar echo intensity and polarization parameters, or simple morphological rules. For example, they extract arc-shaped structures by setting fixed thresholds for differential reflectivity and reflectivity fields, and then combine these with empirical shape parameters such as curvature and length for selection. These methods depend on manually preset physical features. In the data preprocessing stage, clutter filtering and interpolation smoothing are often performed first, followed by using edge detection operators or region growing algorithms to delineate candidate arc segments, and then performing binary classification using statistical classifiers (such as support vector machines or shallow neural networks).

[0003] Conventional approaches suffer from two significant drawbacks. First, the feature extraction process heavily relies on prior knowledge. Differential reflectivity arcs in radar echoes often appear as discontinuous, unevenly intensified arc-shaped bands. Fixed thresholds and manually crafted features struggle to adapt to complex morphological changes under varying weather conditions, leading to high rates of missed or false detections. Second, existing classifiers are mostly constructed based on independent feature spaces, neglecting nonlinear interactions between features, such as the relationship between differential reflectivity and the local gradient of the reflectivity field, and the spatial contextual information of the arc-shaped region. These higher-order correlations are often simplified or discarded in traditional models, limiting the model's ability to capture implicit discriminative cues. Furthermore, single-classifier structures exhibit insufficient generalization performance when dealing with multi-scale arc-shaped features, are susceptible to noise or atypical sample interference, and struggle to maintain stable recognition accuracy in real-time operational systems. Summary of the Invention

[0004] This invention provides a random forest modeling method and system for automatic ZDR arc identification, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a random forest modeling method for automatic ZDR arc identification, comprising: Acquire dual-polarization radar observation data and extract differential reflectivity field and reflectivity field; Based on the spatial relationship between the differential reflectivity field and the reflectivity field, the arc-shaped candidate region is determined, and the corresponding multidimensional feature parameters are extracted. Calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, select feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. The multidimensional feature parameters and higher-order interaction features are used to construct a training sample set; Initialize the basic random forest and construct the decision tree. Calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the decision tree growth and obtain the basic classifier. Extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The features of the data to be identified are input into the base classifier to obtain the discrimination path sequence. The path embedding vector corresponding to the discrimination path sequence is input into the meta-random forest classifier, and the differential reflectance arc recognition result is output.

[0006] In one optional embodiment, calculating the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, filtering feature pairs that simultaneously meet preset threshold conditions, and performing tensor product operations to flatten and generate higher-order interactive features includes: Select any two different features from the multidimensional feature parameters to form a feature pair, calculate the joint probability distribution and marginal probability distribution of the feature pair, calculate the mutual information value based on the joint probability distribution and marginal probability distribution, and organize the mutual information values ​​of all feature pairs in the multidimensional feature parameters into a mutual information matrix according to the row and column indices. Linear regression is performed on the first feature of the feature pair to obtain the remaining features, and linear regression is performed on the second feature of the feature pair to obtain the remaining features, and the correlation coefficient between the first residual vector and the second residual vector is calculated to determine the partial correlation coefficient. The partial correlation coefficients of all feature pairs in the multidimensional feature parameters are organized into a partial correlation coefficient matrix according to the row and column indices. Traverse each position of the mutual information matrix and the partial correlation coefficient matrix. When the mutual information value at that position is greater than the preset mutual information threshold and the absolute value of the partial correlation coefficient at that position is greater than the preset partial correlation threshold, extract the feature pair corresponding to that position and form a set of filtered feature pairs from the extracted feature pairs. For each feature pair in the set of filtered feature pairs, extract the corresponding two feature vectors, perform tensor product operation to generate a higher-order interaction feature tensor, and flatten the higher-order interaction feature tensor into a one-dimensional vector to form a higher-order interaction feature.

[0007] In one optional embodiment, any two different features are selected from the multidimensional feature parameters to form a feature pair. The joint probability distribution and marginal probability distribution of the feature pair are calculated. The mutual information value is calculated based on the joint probability distribution and marginal probability distribution. The mutual information values ​​of all feature pairs in the multidimensional feature parameters are organized into a mutual information matrix according to row and column indices, including: Traverse any two different features in the multidimensional feature parameters to form a feature pair. Calculate the interquartile range for the first and second features in the feature pair. Determine the adaptive binning width based on the interquartile range and the number of samples. Generate the first feature binning boundary sequence and the second feature binning boundary sequence based on the adaptive binning width. A two-dimensional binning grid is constructed based on the first feature binning boundary sequence and the second feature binning boundary sequence. Sample points are mapped to grid cells, and a two-dimensional frequency matrix is ​​constructed by statistically analyzing the frequencies. The joint probability distribution matrix is ​​then obtained by normalization. The first feature marginal probability vector is obtained by summing the joint probability distribution matrix along the row direction, and the second feature marginal probability vector is obtained by summing the joint probability distribution matrix along the column direction. Traverse the elements greater than zero in the joint probability distribution matrix, extract the corresponding index positions, read the marginal probability values ​​of the corresponding index positions from the two marginal probability vectors, and calculate the product to determine the joint probability of the independent hypothesis. Calculate the log ratio based on the elements of the joint probability distribution matrix and the joint probability of the independent hypothesis. Weight the log ratio with the elements of the joint probability distribution matrix to obtain the mutual information contribution of the grid cells. Accumulate the mutual information contributions of all grid cells to obtain the mutual information value of the feature pair. The mutual information matrix is ​​constructed by filling all feature pairs with their mutual information values ​​according to their row and column indices.

[0008] In one optional embodiment, a base random forest is initialized and a decision tree is constructed. The feature conditional entropy of samples at each node of the decision tree is calculated. The sampling probability distribution of each feature is reconstructed, and splitting features are selected to complete the decision tree growth, resulting in a base classifier, including: Initialize the basic random forest and construct a decision tree. Traverse each node of the decision tree. For the sample set of the current node, extract all values ​​of each candidate feature to form a value set. For each value in the value set, select the corresponding sample subset. Count the number of samples of each category in the sample subset to construct a category frequency vector. Calculate the information entropy based on the category frequency vector to obtain the conditional entropy corresponding to the value. Sum the conditional entropies of all values ​​of the candidate feature according to the sample subset size to obtain the feature conditional entropy. The reciprocal of the feature conditional entropy of all candidate features is taken element by element to obtain the entropy reciprocal vector. The entropy reciprocal vector is converted into a probability distribution to obtain the sampling probability vector as the sampling probability distribution of each feature. A dynamic sampling probability vector is constructed based on the sampling probability distribution and the historical splitting frequency. Weighted random sampling is performed on the candidate features to obtain the initial selection features. The splitting features are obtained based on the initial selection features. Based on the optimal split point of the splitting feature, the current node sample set is split to generate left and right child nodes. The node splitting and feature sampling are recursively performed until the preset maximum depth or preset minimum number of samples is met, and the decision tree growth is completed to obtain the basic classifier.

[0009] In one optional embodiment, a dynamic sampling probability vector is constructed based on the sampling probability distribution and historical splitting frequency. Weighted random sampling is then performed on the candidate features to obtain initial selected features. Split features are then obtained based on these initial selected features, including: Extract the sampling probability value corresponding to each candidate feature from the sampling probability vector, calculate the ratio of the sampling probability value to the historical split frequency for each candidate feature to obtain the adjustment factor, multiply the sampling probability value by the adjustment factor to determine the dynamic sampling probability, and construct a dynamic sampling probability vector for the dynamic sampling probabilities of all candidate features. Calculate the cumulative sum of the dynamic sampling probability vector to construct the cumulative probability vector, generate random numbers that follow a uniform distribution, find the position index of the first position greater than the random number in the cumulative probability vector, and extract the candidate features corresponding to the position index as the initial selection features. Calculate the information gain of the initial selected feature on the current node sample set. If the information gain is greater than the preset information gain threshold, the initial selected feature is determined as the splitting feature. Otherwise, the initial selected feature is removed from the candidate features and dynamic sampling is re-executed until the splitting feature is obtained. The historical splitting frequency of the splitting characteristics is incremented and the historical splitting frequency record is updated.

[0010] In one optional embodiment, the discriminant path sequences of each decision tree in the base classifier are extracted, the path sequences are converted into a topological graph and encoded into path embedding vectors, and a meta-random forest classifier is trained based on the path embedding vectors, including: Traverse each decision tree in the base classifier from the root node to the leaf node, extract the node identifiers and split features passed during the traversal to form a discrimination path sequence, map the node identifiers and split features in the discrimination path sequence to node attributes, map the parent-child relationship of adjacent nodes to directed edges, and construct the topology graph of the discrimination path based on the node attributes and directed edges. Initialize node feature vectors for each node in the topology graph. Use a message passing mechanism to perform feature aggregation layer by layer from leaf node to root node. For the current node, collect the node feature vectors of its child nodes and perform weighted summation to obtain the aggregated feature vector. Concatenate the aggregated feature vector with the depth position feature and split feature index of the current node to obtain the updated node feature vector. Extract the node feature vector of the root node as the path embedding vector. The path embedding vectors corresponding to all decision trees and the classification accuracy of each decision tree on the validation set are combined into a meta-training sample set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. All meta-decision trees are aggregated to obtain the meta-random forest classifier.

[0011] In one optional embodiment, a meta-training sample set is formed by combining the path embedding vectors corresponding to all decision trees with the classification accuracy of each decision tree on the validation set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. The meta-random forest classifier is obtained by aggregating all meta-decision trees, including: The path embedding vectors corresponding to all decision trees are used to form a meta-feature matrix according to the decision tree index. The classification accuracy of each decision tree on the validation set is used to form a meta-label vector according to the decision tree index. The corresponding elements of each row vector of the meta-feature matrix and the meta-label vector are used to form a meta-training sample pair and a meta-training sample set. Multiple rounds of random sampling with replacement are performed on the meta-training sample set to obtain multiple meta-bootstrap sample sets. For each meta-bootstrap sample set, a corresponding meta-decision tree is constructed. The meta-training sample pairs that were not sampled in the meta-training sample set are used to form an out-of-bag sample set. The prediction error of each meta-decision tree on the corresponding out-of-bag sample set is calculated. Meta-decision trees with prediction errors lower than a preset error threshold are selected and retained. The retained meta-decision trees are aggregated to obtain a meta-random forest classifier. The voting weights of each meta-decision tree are calculated for the meta-random forest classifier. Weighted voting is performed on the prediction results of each meta-decision tree to obtain the ensemble prediction output of the meta-random forest classifier.

[0012] A second aspect of this invention provides a random forest modeling system for automatic ZDR arc identification, comprising: The data acquisition unit is used to acquire dual-polarization radar observation data and extract the differential reflectivity field and reflectivity field. The region identification unit is used to determine the arc-shaped candidate region based on the spatial relationship between the differential reflectance field and the reflectance field, and to extract the corresponding multidimensional feature parameters. The feature extraction unit is used to calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, filter feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. A sample construction unit is used to construct a training sample set by combining the multidimensional feature parameters with higher-order interaction features; The classifier building unit is used to initialize the basic random forest and build a decision tree, calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the growth of the decision tree and obtain the basic classifier. The feature encoding unit is used to extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The result output unit is used to input the features of the data to be identified into the base classifier to obtain the discrimination path sequence, input the path embedding vector corresponding to the discrimination path sequence into the meta-random forest classifier, and output the differential reflectance arc recognition result.

[0013] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] In this embodiment of the invention, by selecting feature pairs whose mutual information and partial correlation coefficients both meet threshold conditions and generating high-order interaction features, the nonlinear coupling relationship between the differential reflectivity field and the reflectivity field in the arc-shaped region is effectively captured, avoiding the missed detections and false detections caused by traditional methods that rely solely on a single feature or linear combination. The path embedding vector transforms the discrimination logic of the decision tree into topological structure features. Based on this, the meta-random forest classifier learns the association patterns between paths, making the recognition results more robust to noise and local data gaps. The sampling probability distribution is reconstructed based on feature conditional entropy, and the selection weights of split features are dynamically adjusted, suppressing the interference of redundant features on the growth of the decision tree and enabling the basic classifier to maintain a stable discrimination boundary in complex weather scenarios. The topological graph encoding preserves the global structural information of the path sequence. The meta-random forest classifier integrates the decision logic of multiple trees through path embedding vectors, avoiding overfitting of a single tree to a specific sample distribution, and maintaining consistent recognition performance across different radar observation periods and geographical regions; a balance is achieved between computational efficiency and interpretability. The dual screening mechanism of mutual information matrix and partial correlation coefficient matrix retains only strongly correlated feature pairs for tensor product operation, keeping the dimensionality of high-order interaction features within a reasonable range and reducing computational overhead during training. The path embedding vector directly reflects the discrimination criteria of the decision tree, and the output of the meta-random forest classifier can be traced back to a specific path sequence, facilitating the analysis of the contribution of each feature parameter during arc recognition and providing a clear basis for radar data post-processing. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the random forest modeling method for automatic ZDR arc identification; Figure 2 Flowchart for constructing a meta-random forest classifier. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0019] Figure 1 This is a flowchart illustrating the random forest modeling method for automatic ZDR arc identification according to an embodiment of the present invention, as shown below. Figure 1 As shown, the random forest modeling method for automatic ZDR arc identification includes: Acquire dual-polarization radar observation data and extract differential reflectivity field and reflectivity field; Based on the spatial relationship between the differential reflectivity field and the reflectivity field, the arc-shaped candidate region is determined, and the corresponding multidimensional feature parameters are extracted. Calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, select feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. The multidimensional feature parameters and higher-order interaction features are used to construct a training sample set; Initialize the basic random forest and construct the decision tree. Calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the decision tree growth and obtain the basic classifier. Extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The features of the data to be identified are input into the base classifier to obtain the discrimination path sequence. The path embedding vector corresponding to the discrimination path sequence is input into the meta-random forest classifier, and the differential reflectance arc recognition result is output.

[0020] In one optional embodiment, calculating the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, filtering feature pairs that simultaneously meet preset threshold conditions, and performing tensor product operations to flatten and generate higher-order interactive features includes: Select any two different features from the multidimensional feature parameters to form a feature pair, calculate the joint probability distribution and marginal probability distribution of the feature pair, calculate the mutual information value based on the joint probability distribution and marginal probability distribution, and organize the mutual information values ​​of all feature pairs in the multidimensional feature parameters into a mutual information matrix according to the row and column indices. Linear regression is performed on the first feature of the feature pair to obtain the remaining features, and linear regression is performed on the second feature of the feature pair to obtain the remaining features, and the correlation coefficient between the first residual vector and the second residual vector is calculated to determine the partial correlation coefficient. The partial correlation coefficients of all feature pairs in the multidimensional feature parameters are organized into a partial correlation coefficient matrix according to the row and column indices. Traverse each position of the mutual information matrix and the partial correlation coefficient matrix. When the mutual information value at that position is greater than the preset mutual information threshold and the absolute value of the partial correlation coefficient at that position is greater than the preset partial correlation threshold, extract the feature pair corresponding to that position and form a set of filtered feature pairs from the extracted feature pairs. For each feature pair in the set of filtered feature pairs, extract the corresponding two feature vectors, perform tensor product operation to generate a higher-order interaction feature tensor, and flatten the higher-order interaction feature tensor into a one-dimensional vector to form a higher-order interaction feature.

[0021] In one specific implementation, after obtaining the multidimensional feature parameters, it is necessary to perform quantitative analysis on the correlation between the features in order to discover feature pairs with strong interactions, and generate interactive features that can capture high-order nonlinear relationships through tensor product operations, thereby enhancing the discriminative ability of the subsequent random forest model.

[0022] For the extracted multidimensional feature parameters, any two distinct features are selected to form a feature pair. Let the multidimensional feature parameters contain a total of... If there are features, then the total number of feature pairs is . For each feature pair, the marginal probability distributions of each feature and their joint probability distribution are calculated. Specifically, for the discretized feature value intervals, the frequency of the sample within each interval is used as a probability estimate. Let the feature be... The marginal probability distribution is ,feature The marginal probability distribution is The joint probability distribution of the two is Then the feature pair The mutual information value is calculated as follows: ; in, and Traversing features separately and All discretized value ranges. Mutual information value The larger the mutual information value, the more information the two features share and the higher their interdependence. When two features are completely independent, the mutual information value is zero. The mutual information matrix is ​​constructed by arranging the mutual information values ​​of all feature pairs according to their row and column indices. , its first Each element is a feature. With features The mutual information values ​​between them.

[0023] Mutual information can capture non-linear dependencies between features, but it cannot distinguish between direct associations between two features and indirect associations generated through a third feature. Therefore, a partial correlation coefficient is further introduced to measure the net linear correlation between feature pairs after excluding the influence of other features. For feature pairs... , will feature As the dependent variable, with the rest Using these features as independent variables to perform linear regression, we obtain... The first residual vector for the remaining features ; Features As the dependent variable, with the same remainder Using these features as independent variables to perform linear regression, we obtain... The second residual vector regarding the remaining features First residual vector represent Independent variation components that cannot be explained by other features, the second residual vector represent Independent variation components that cannot be explained by other features. Calculation and The Pearson correlation coefficient, which yields the feature pairs, is obtained. partial correlation coefficient : ; in, For the sample size, and The first One sample in and The residual value in and These are the means of the corresponding residual vectors. The partial correlation coefficients of all feature pairs are arranged according to the row and column indices of the features to form a partial correlation coefficient matrix. , its first The element is .

[0024] After obtaining the mutual information matrix With partial correlation coefficient matrix Then, iterate through each corresponding position in both matrices. Simultaneously test two conditions: position Is the mutual information value at the location greater than the preset mutual information threshold? and location Does the absolute value of the partial correlation coefficient exceed the preset partial correlation threshold? Only when both conditions are met, i.e. and Only then will the feature pair corresponding to that position be... Extracted and added to the filter feature pair set Preset mutual information threshold The settings can be referenced from the overall distribution of feature pair mutual information values, selecting values ​​at higher quantiles to retain feature pairs with significant information content; a preset partial correlation threshold can be used. The threshold is typically set between 0.3 and 0.5 to filter feature pairs with substantial net linear correlation. This dual threshold filtering effectively excludes feature pairs with only indirect or weak correlations, ensuring that the feature pairs entering subsequent processing have strong direct interactions.

[0025] For the set of filtering features Each feature pair in Extract the two corresponding feature vectors. Let the feature vectors be... The corresponding feature vector is ,feature The corresponding feature vector is ,in and These are the dimensions of the two feature vectors. and Perform tensor product operations to generate higher-order interactive feature tensors. : ; The result of the tensor product operation For one The matrix whose first... The elements are ,Right now The Each component and The The product of components. Tensor product operations can explicitly construct all pairwise product combinations between two features, thereby capturing the multiplicative interaction effects between features in the feature space. This is of great significance for describing the nonlinear coupling relationship between radar echo features in differential reflectivity arcs. High-order interactive feature tensors... Flattening it into a one-dimensional vector in row-major order, we get a vector of dimension [dimensional value]. High-order interactive feature vectors .

[0026] For the set of filter feature pairs All feature pairs are subjected to the tensor product and flattening operations described above, and all the resulting higher-order interactive feature vectors are concatenated together to form a complete higher-order interactive feature representation. Finally, the original multidimensional feature parameter vector and the higher-order interactive feature vector are concatenated to form the enhanced feature representation, which is used to construct the subsequent training sample set. This feature enhancement method not only retains the independent discriminative information of the original features, but also introduces a description of the synergistic effect between features through higher-order interactive features, which can provide richer discriminative criteria for the random forest model and help improve the recognition accuracy of differential reflectivity arcs.

[0027] In one optional embodiment, any two different features are selected from the multidimensional feature parameters to form a feature pair. The joint probability distribution and marginal probability distribution of the feature pair are calculated. The mutual information value is calculated based on the joint probability distribution and marginal probability distribution. The mutual information values ​​of all feature pairs in the multidimensional feature parameters are organized into a mutual information matrix according to row and column indices, including: Traverse any two different features in the multidimensional feature parameters to form a feature pair. Calculate the interquartile range for the first and second features in the feature pair. Determine the adaptive binning width based on the interquartile range and the number of samples. Generate the first feature binning boundary sequence and the second feature binning boundary sequence based on the adaptive binning width. A two-dimensional binning grid is constructed based on the first feature binning boundary sequence and the second feature binning boundary sequence. Sample points are mapped to grid cells, and a two-dimensional frequency matrix is ​​constructed by statistically analyzing the frequencies. The joint probability distribution matrix is ​​then obtained by normalization. The first feature marginal probability vector is obtained by summing the joint probability distribution matrix along the row direction, and the second feature marginal probability vector is obtained by summing the joint probability distribution matrix along the column direction. Traverse the elements greater than zero in the joint probability distribution matrix, extract the corresponding index positions, read the marginal probability values ​​of the corresponding index positions from the two marginal probability vectors, and calculate the product to determine the joint probability of the independent hypothesis. Calculate the log ratio based on the elements of the joint probability distribution matrix and the joint probability of the independent hypothesis. Weight the log ratio with the elements of the joint probability distribution matrix to obtain the mutual information contribution of the grid cells. Accumulate the mutual information contributions of all grid cells to obtain the mutual information value of the feature pair. The mutual information matrix is ​​constructed by filling all feature pairs with their mutual information values ​​according to their row and column indices.

[0028] In one specific implementation, the process of constructing the mutual information matrix requires iterating through all pairwise combinations of features in the multidimensional feature parameters, and performing probability distribution estimation and mutual information calculation for each feature pair. Unlike directly using a fixed bin width, this approach employs an adaptive binning strategy based on quartile spacing to accommodate the differences in numerical distributions of different features, avoiding binning distortion caused by distribution skewness or outliers.

[0029] For any feature pair, extract the observation sequences of the first and second features across all samples. For the first feature, calculate the difference between its 75th percentile and 25th percentile to obtain the interquartile range of the first feature. Similarly, the interquartile range of the second feature is calculated. The interquartile interval is insensitive to extreme values ​​and can robustly reflect the dispersion of data; therefore, it is used as the basis for calculating bin width.

[0030] When determining the binning width, the Freedman-Diaconis criterion is followed, and the binning width is set as a function of the interquartile range and the number of samples. Specifically, the adaptive binning width for the first feature... Adaptive binning width with the second feature They are determined in the following ways: ; ; in The number of samples. or When the value is zero (e.g., a feature has a highly concentrated value), a backup bin width based on the standard deviation can be used instead to prevent the number of bins from degenerating to infinity.

[0031] Based on the minimum and maximum values ​​of the first feature observation, combined with the bin width Generate the bin boundary sequence of the first feature, that is... Interval by step size Uniform division yields a set of monotonically increasing boundary values; similarly, based on the numerical range of the second feature and the bin width... Generate a binning boundary sequence for the second feature. The two binning boundary sequences together define a two-dimensional binning grid, where the number of rows is determined by the number of bins in the first feature and the number of columns is determined by the number of bins in the second feature.

[0032] The first and second eigenvalues ​​of each sample point are mapped to their corresponding binning intervals, determining their row and column indices in a two-dimensional grid, thus assigning each sample point to a unique grid cell. After mapping all samples, the number of samples falling into each grid cell is counted, constructing a two-dimensional frequency matrix. The first in the matrix Line 1 Column elements This indicates that the first feature falls into the first... The first box and the second feature falls into the first The number of samples per bin. Divide by the total number of samples After normalization, the joint probability distribution matrix is ​​obtained. Its elements This represents the empirical joint probability of a sample within that grid cell.

[0033] For the joint probability distribution matrix Summing along the column direction (i.e., summing over all column elements in each row) yields the marginal probability vector of the first feature. , its first element This indicates that the first feature falls into the first... The marginal probability of each bin; for Summing along the row direction (i.e., over all row elements in each column) yields the marginal probability vector of the second feature. , its first element This indicates that the second feature falls into the first... The marginal probabilities of each bin. The marginal probability vector satisfies the normalization condition, i.e., the sum of all elements is 1.

[0034] When calculating mutual information values, only the joint probability distribution matrix is ​​traversed. Elements greater than zero are included to avoid numerical anomalies caused by zero values ​​in logarithmic operations. For elements satisfying... Grid cells From the marginal probability vector Read from ,from Read from Calculate the product of the two. This is the joint probability of the grid cell under the independence assumption. The actual joint probability... Joint probability with independence assumption Taking the natural logarithm of the ratio yields the logarithmic ratio. Then compare the logarithmic ratio with Multiplying them together gives the contribution of the grid cell to the mutual information. : ; All satisfied The contribution of the grid cells By summing the results, we obtain the mutual information value of the feature pair. : ; The unit of measurement is nat. To convert it to bit, divide by . Mutual information value Non-negative, takes the value of zero if and only if the two features are statistically independent, and the larger the value, the stronger the statistical dependence between the two features.

[0035] The above calculation process is performed sequentially on all unique feature pairs in the multidimensional feature parameters to obtain the mutual information value corresponding to each feature pair. These mutual information values ​​are then filled into the matrix according to the row and column indices of the features. In the middle: If the first The first feature and the second The mutual information value of each feature is Then fill it in. The Line 1 Column and number Line 1 The column (mutual information is symmetrical) has diagonal elements corresponding to the mutual information between the feature and itself, which can be set as the entropy value of the feature itself or as a predefined constant. The final result... For one A symmetric matrix, where The total number of features in the multidimensional feature parameters is represented by the matrix. Each off-diagonal element in the matrix fully characterizes the nonlinear statistical correlation strength between the corresponding feature pairs.

[0036] The advantages of adaptive binning strategies compared to fixed-width binning are as follows: for features with small variance and highly concentrated values, a finer binning granularity is automatically adopted to preserve local distribution details; for features with wide numerical ranges and high dispersion, the binning width is automatically increased to avoid the frequency matrix becoming too sparse, leading to unstable probability estimation. This adaptive mechanism makes the mutual information matrix... The estimated values ​​in this study maintain high statistical reliability across different feature scales, providing a basis for subsequent studies based on... The characteristics of the threshold provide an accurate quantitative basis for screening.

[0037] When processing multidimensional feature parameters extracted from dual-polarization radar observation data, the numerical distributions of different features vary significantly, such as differential reflectivity. Dynamic range and reflectivity factor The dynamic ranges of these features vary considerably, while the distribution of texture or shape features tends to be more concentrated. Adaptive binning widths can automatically adjust to the actual distribution of each feature, thereby ensuring the comparability of mutual information estimation results for all feature pairs and supporting the reliable generation of subsequent higher-order interactive features.

[0038] In one optional embodiment, a base random forest is initialized and a decision tree is constructed. The feature conditional entropy of samples at each node of the decision tree is calculated. The sampling probability distribution of each feature is reconstructed, and splitting features are selected to complete the decision tree growth, resulting in a base classifier, including: Initialize the basic random forest and construct a decision tree. Traverse each node of the decision tree. For the sample set of the current node, extract all values ​​of each candidate feature to form a value set. For each value in the value set, select the corresponding sample subset. Count the number of samples of each category in the sample subset to construct a category frequency vector. Calculate the information entropy based on the category frequency vector to obtain the conditional entropy corresponding to the value. Sum the conditional entropies of all values ​​of the candidate feature according to the sample subset size to obtain the feature conditional entropy. The reciprocal of the feature conditional entropy of all candidate features is taken element by element to obtain the entropy reciprocal vector. The entropy reciprocal vector is converted into a probability distribution to obtain the sampling probability vector as the sampling probability distribution of each feature. A dynamic sampling probability vector is constructed based on the sampling probability distribution and the historical splitting frequency. Weighted random sampling is performed on the candidate features to obtain the initial selection features. The splitting features are obtained based on the initial selection features. Based on the optimal split point of the splitting feature, the current node sample set is split to generate left and right child nodes. The node splitting and feature sampling are recursively performed until the preset maximum depth or preset minimum number of samples is met, and the decision tree growth is completed to obtain the basic classifier.

[0039] In one specific implementation, when constructing the basic random forest, the forest is first initialized by setting hyperparameters such as the number of decision trees, maximum depth, and minimum number of samples per node, and an independent bootstrap sampling dataset is assigned to each decision tree. For each decision tree, each node is traversed layer by layer starting from the root node, and the feature conditional entropy calculation process is performed on the sample set held by the current node.

[0040] For the sample set of the current node, iterate through all candidate features. For a given candidate feature... Extract the values ​​of this feature across all samples of the current node to form a value set. The construction method of the value set varies depending on the feature type: for discrete features, all the values ​​that have appeared are directly enumerated; for continuous features, they are discretized into several bins, with each bin corresponding to a value.

[0041] For the set of values Each value in Filter from the current node's sample set to find samples where the feature value is equal to (or a subset of samples falling into the corresponding binning interval) ,statistics The number of samples in each category is used to construct a category frequency vector. Let the total number of categories be... Category frequency vector The Each component is denoted as , indicating in the sample subset The middle belongs to the first The number of samples in each category, and the total size of the sample subset is denoted as . .

[0042] Based on category frequency vector Calculate the sample subset Information entropy Let the first... The proportion of each category in the sample subset is Then the information entropy is: ; Among them, when At that time, it was agreed This information entropy reflects the characteristics Values Under certain conditions, the degree of uncertainty regarding the category of the sample set.

[0043] For candidate features All possible values, according to the size of each sample subset Total number of samples at the current node The weighted sum of the ratios yields the characteristic Feature conditional entropy : ; Feature conditional entropy The smaller the value, the purer the category distribution within the child nodes after partitioning the current node using this feature, and the greater the contribution of this feature to the classification of the current node. Perform the above calculation sequentially on all candidate features to obtain the feature conditional entropy corresponding to each feature in the candidate feature set.

[0044] After calculating the feature conditional entropy of all candidate features, the reciprocal of the feature conditional entropy of each candidate feature is taken element by element to construct the entropy reciprocal vector. Let the total number of candidate features be... , No. The feature conditional entropy of each candidate feature is: Then the first inverse of the entropy vector Each component is When the conditional entropy of a certain feature is extremely small, a very small smoothing constant can be added to the denominator to prevent numerical overflow. ,Right now .

[0045] The inverse entropy vector Normalization converts the sampled probability vector into a probability distribution. , its first The components are: ; Sampling probability vector This reflects the initial tendency of each candidate feature to be selected for splitting at the current node; features with smaller conditional entropy are assigned a higher sampling probability.

[0046] Based on this, historical split frequency is introduced to dynamically adjust the sampling probability, in order to avoid the overuse of certain features in the entire decision tree, which would lead to a decrease in the diversity of the tree structure. Let the feature be... The cumulative number of times a node has been selected as a splitting feature among all nodes that have completed splitting in the current decision tree is: The historical split frequency vector is denoted as Then the dynamic sampling probability vector The Each component is obtained by combining the sampling probability vector with the attenuation factor of the historical frequency: ; in, This is the historical frequency penalty coefficient, used to control the degree to which historical split frequencies suppress the sampling probability. When the value is large, the probability of features that have been frequently selected in history being selected in subsequent nodes is significantly reduced, thereby promoting the balanced use of different features in the tree. When the value is small, the effect of dynamic adjustment is close to that of static sampling that relies solely on the feature conditional entropy.

[0047] Based on the dynamic sampling probability vector Weighted random sampling is performed on the candidate features to obtain a preliminary feature subset. The size of the initial feature subset is typically set to the square root or logarithm of the total number of candidate features, consistent with the feature sampling size of a standard random forest. In the initial feature subset... In the next step, the information gain or Gini impurity reduction of each feature is calculated, and the feature that maximizes the purity improvement of the current node is selected as the final splitting feature. .

[0048] Determine the splitting characteristics Then, iterate through all candidate split points for this feature, calculate the sum of weighted information entropy of the left and right child nodes after splitting for each candidate split point, and select the split point that minimizes the sum of weighted information entropy as the optimal split point. Based on the optimal split point Divide the sample set of the current node into the sample set of its left child nodes. (satisfy (sample of the first child node) and the set of samples of the right child node (satisfy (from the sample), complete the splitting operation of the current node, and generate the left child node and the right child node.

[0049] The process involves recursively performing the aforementioned feature conditional entropy calculation, dynamic sampling probability update, weighted random sampling, optimal splitting feature and splitting point determination, and node splitting on both the left and right child nodes. The recursion terminates when the current node depth reaches a preset maximum depth. Or the number of samples in the current node is less than the preset minimum number of samples. If any of the above stopping conditions are met, the current node is designated as a leaf node, and the category with the highest percentage in the sample set of that node is used as the predicted category for the leaf node.

[0050] The above construction process is performed independently on each decision tree in the random forest. Each decision tree exhibits structural differences under a dual randomization mechanism of bootstrapping sampled datasets and dynamic sampling probabilities, thus ensuring the diversity of the entire forest. After all decision trees are built, they are aggregated into a base classifier, which predicts the feature vectors of the input samples. Each decision tree independently outputs its predicted class, and the final classification result is determined by majority voting. This base classifier balances feature discrimination ability with tree structure diversity, providing a rich and discriminative set of decision trees for subsequent path embedding in the meta-random forest.

[0051] In one optional embodiment, a dynamic sampling probability vector is constructed based on the sampling probability distribution and historical splitting frequency. Weighted random sampling is performed on the candidate features to obtain initial selected features. Split features are then obtained based on the initial selected features, including: Extract the sampling probability value corresponding to each candidate feature from the sampling probability vector, calculate the ratio of the sampling probability value to the historical split frequency for each candidate feature to obtain the adjustment factor, multiply the sampling probability value by the adjustment factor to determine the dynamic sampling probability, and construct the dynamic sampling probability vector for all candidate features. Calculate the cumulative sum of the dynamic sampling probability vector to construct the cumulative probability vector, generate random numbers that follow a uniform distribution, find the position index of the first position greater than the random number in the cumulative probability vector, and extract the candidate features corresponding to the position index as the initial selection features. Calculate the information gain of the initial selected feature on the current node sample set. If the information gain is greater than the preset information gain threshold, the initial selected feature is determined as the splitting feature. Otherwise, the initial selected feature is removed from the candidate features and dynamic sampling is re-executed until the splitting feature is obtained. The historical splitting frequency of the splitting characteristics is incremented and the historical splitting frequency record is updated.

[0052] In one specific implementation, after constructing the sampling probability vectors of candidate features, it is necessary to further introduce historical splitting frequency information to dynamically adjust the sampling probabilities, thereby avoiding the repeated selection of certain features during the decision tree growth process, which could lead to homogenization of the tree structure. Specifically, for each candidate feature... From the constructed sampling probability vector Extract the corresponding sampling probability value Simultaneously, read the historical splitting frequency of this feature. .Will and Perform a ratio calculation to obtain the adjustment factor. ,Right now ,in This is a numerical smoothing constant to prevent division-to-zero anomalies when the historical splitting frequency is zero. Adjustment factor The physical meaning is: if a certain feature's history is selected a large number of times, then... Larger The corresponding reduction will suppress the feature in subsequent sampling; if a feature has rarely been selected historically, then... A larger value helps increase the probability of that feature being selected, thereby promoting the diversity of the decision tree.

[0053] Will and Multiply to get the first Dynamic sampling probability of each candidate feature ,Right now For all After performing the above calculations on each candidate feature, all Normalization, constructing a dynamic sampling probability vector The normalization formula is This ensures that the sum of all components is 1, satisfying the basic constraints of the probability distribution. Dynamic sampling probability vector. This comprehensively reflects both the discriminative power of the features and their historical usage frequency, serving as the core basis for subsequent weighted random sampling.

[0054] In the dynamic sampling probability vector After construction, weighted random sampling is implemented using the cumulative probability vector. The components are summed sequentially to construct a cumulative probability vector. , its first Each component is The last component of the cumulative probability vector is always 1 after normalization, forming a complete probability interval partition. This generates a probability vector that follows a uniform distribution. random numbers In the cumulative probability vector Perform a binary search to locate the first matching condition. Location index Extract candidate features corresponding to the position index. As a preliminary selection feature This process is essentially an inverse transformation sampling, where features with higher dynamic sampling probabilities correspond to a wider probability interval. Random numbers that fall within this interval are selected, thus achieving weighted random sampling.

[0055] Obtaining initial selection features Next, it is necessary to further verify whether it possesses sufficient splitting ability. On the sample set of the current node, the information gain of the initially selected features is calculated. Information gain measures the improvement in the purity of the sample set after splitting a node using a specific feature. Specifically, it first calculates the overall information entropy of the current node. Then, iterate through all possible values ​​of the initially selected features, and sum them according to the sample proportion of the subset corresponding to each value to obtain the conditional entropy. The difference between the two is the information gain. .when Greater than the preset information gain threshold At that time, if the initially selected features are deemed to make a substantial contribution to the classification of the current node, they are determined as the final splitting features. And select the split point that maximizes information gain from all possible split points. Based on this, the sample set of the current node is divided into the sample set of its left child nodes. With the right child node sample set .

[0056] when The preset information gain threshold is not met. When describing the initial selection characteristics If the feature's contribution to the current node's classification is insufficient, it cannot be considered a valid splitting feature. In this case, the initially selected feature is temporarily removed from the candidate feature set for the current round, and the dynamic sampling process is re-executed: the dynamic sampling probability vector is re-normalized based on the remaining candidate features, uniform random numbers are generated again, and new initially selected features are searched. The information gain verification steps described above are repeated until a splitting feature that meets the conditions is found. If the candidate feature set is exhausted after multiple rounds of removal and no feature meeting the conditions is found, the current node is set as a leaf node, and the majority class of the samples within the node is used as the prediction output, terminating further growth of this branch. This mechanism effectively prevents overfitting caused by forcibly splitting in regions with complex sample distributions.

[0057] In determining the characteristics of splitting Afterwards, Frequency of historical divisions Perform an increment operation, i.e. And write the updated value back to the historical split frequency vector. Historical split frequencies are continuously maintained throughout the growth of the decision tree, updated synchronously with each node split. In this way, as the decision tree grows, the historical split frequencies corresponding to frequently selected features accumulate, and their dynamic sampling probabilities are gradually reduced in subsequent node splits, while historically less frequently selected features gain more opportunities for exploration. This dynamic feedback mechanism results in greater diversity in feature selection paths across the decision trees in the random forest, thereby achieving stronger generalization ability during the ensemble classification stage through majority voting or probability averaging.

[0058] In ZDR arc recognition scenarios, multidimensional feature parameters encompass various types of features, including differential reflectivity local gradient, reflectivity intensity distribution, and arc geometry descriptors. Some of these features exhibit strong spatial correlation. Without a dynamic adjustment mechanism based on historical split frequencies, highly correlated features are often repeatedly selected due to their similar discriminative abilities, leading to structural uniformity among multiple decision trees and a degradation in ensemble performance. By constructing and updating the aforementioned dynamic sampling probability vector, the structural differences between trees can be effectively improved while ensuring sufficient discriminative power for each decision tree, ultimately enhancing the accuracy and robustness of automatic ZDR arc recognition.

[0059] like Figure 2 The diagram shows the flowchart for constructing a meta-random forest classifier.

[0060] In one optional embodiment, the discriminant path sequences of each decision tree in the base classifier are extracted, the path sequences are converted into a topological graph and encoded into path embedding vectors, and a meta-random forest classifier is trained based on the path embedding vectors, including: Traverse each decision tree in the base classifier from the root node to the leaf node, extract the node identifiers and split features passed during the traversal to form a discrimination path sequence, map the node identifiers and split features in the discrimination path sequence to node attributes, map the parent-child relationship of adjacent nodes to directed edges, and construct the topology graph of the discrimination path based on the node attributes and directed edges. Initialize the node feature vector for each node in the topology graph. Use a message passing mechanism to perform feature aggregation layer by layer from the leaf node to the root node. For the current node, collect the node feature vectors of its child nodes and perform weighted summation to obtain the aggregated feature vector. Concatenate the aggregated feature vector with the depth position feature and split feature index of the current node to obtain the updated node feature vector. Extract the node feature vector of the root node as the path embedding vector. The path embedding vectors corresponding to all decision trees and the classification accuracy of each decision tree on the validation set are combined into a meta-training sample set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. All meta-decision trees are aggregated to obtain the meta-random forest classifier.

[0061] In one specific implementation, after the base classifier has been trained, structured discriminative information needs to be extracted from each decision tree to further train the meta-random forest classifier. Specifically, each decision tree in the base classifier is traversed, starting from the root node, and each internal node and leaf node is visited sequentially according to the depth-first traversal strategy. The unique identifier of each node traversed on the traversal path and the feature name used when that node performs a split are recorded (leaf nodes do not contain split features, only their identifier and class label are recorded). The above information is arranged in order from root to leaf to form the discriminative path sequence of the decision tree. The discriminative path sequence fully describes the logical structure of the decision tree's step-by-step partitioning in the sample space and is the basis for subsequent graph coding.

[0062] When converting the discriminant path sequence into a topology graph, each node identifier in the sequence is mapped to its corresponding splitting feature as a node attribute in the graph. For internal nodes, the node attributes include the node's depth in the tree, the index number of the splitting feature, and the splitting threshold; for leaf nodes, the node attributes include depth and the predicted class. The parent-child relationships between adjacent nodes in the discriminant path sequence are mapped to directed edges, with the edges pointing from the parent node to the child node, thus constructing a directed acyclic graph, which is the discriminant path topology graph of the decision tree. The number of nodes in the topology graph is the same as the total number of nodes in the decision tree, and the number of edges is equal to the number of nodes minus one.

[0063] In the process of calculating the path embedding vector, the node feature vectors of each node in the topology graph are first initialized. Let the node... The initial node feature vector is Its dimensions are The initialization method involves concatenating node attributes (depth normalized value, one-hot encoding or zero vector of split feature index (leaf node), split threshold normalized value) and then mapping them to a linear transformation. A 3D space is then used. A message-passing mechanism is then employed to perform feature aggregation layer by layer from the leaf nodes to the root node. For the current node... Collect all its child nodes Node feature vectors Perform a weighted summation to obtain the aggregated feature vector: ; in For nodes child nodes Aggregate weights, For nodes The set of child nodes Number the current aggregation level. Weight The weight can be determined based on the sample coverage ratio of the child nodes, that is, the ratio of the number of training samples covered by the child nodes to the total number of samples covered by the current node is used as the weight, so that the child nodes with a larger sample size contribute more information to the aggregation result.

[0064] Aggregate feature vectors With the current node Depth location features Embedded representation of split feature index The concatenated vectors are then processed through linear transformation and activation functions to obtain the updated node feature vectors. : ; in For the first The linear transformation weight matrix of the layer, For the corresponding bias vector, For activation functions (such as ReLU). This represents a vector concatenation operation. Message passing starts at the leaf node level and propagates upwards to the root node, performing aggregation rounds equal to the tree depth. After aggregation at all levels, the final node feature vector of the root node is extracted. As the path embedding vector of this decision tree, denoted as Its dimensions are The path embedding vector integrates all structural information of the decision tree from root to leaf, including the distribution of splitting features at each level, depth positional relationships, and sample coverage ratio, and can effectively characterize the discriminative ability of the decision tree.

[0065] During the training phase of the meta-random forest classifier, the path embedding vectors corresponding to all decision trees in the base classifier are aligned with the classification accuracy of each decision tree on the validation set according to the decision tree index, forming a meta-training sample set. Each sample in the meta-training sample set is represented by a path embedding vector. With the corresponding classification accuracy Composition, in which This refers to the index number of the decision tree. Classification accuracy. By evaluating the first on the independent validation set The consistency ratio between the prediction results of each decision tree and the true labels is obtained, and the validation set does not overlap with the training set to ensure the objectivity of the evaluation results.

[0066] Bootstrap sampling is performed on the meta-training sample set. This involves randomly selecting a subset of samples with replacement from the meta-training sample set, creating a bootstrap subset, and training a meta-decision tree based on this subset. The training objective of the meta-decision tree is a regression task, using path embedding vectors as input features and classification accuracy as the regression objective. Mean squared error is used as the optimization criterion for node splitting. During the node splitting process of the meta-decision tree, samples are randomly selected from the path embedding vectors... Select from the dimensions Each dimension is used as a candidate splitting feature. , usually take (Rounding down to the square root), iterate through the range of candidate dimensions, selecting the dimension and split point that minimizes the mean squared error, and perform the split. Repeat the above bootstrap sampling and meta-decision tree construction process to build a total of... The prediction results of all the meta-decision trees are averaged to obtain an estimate of the prediction accuracy of the meta-random forest classifier for each base decision tree.

[0067] In the inference phase, features are extracted from the data to be identified and input into the base classifier. Each decision tree outputs a discriminative path sequence, which is then aggregated through topology graph construction and message passing to obtain path embedding vectors. These path embedding vectors are then input into a meta-random forest classifier to obtain an estimate of the prediction accuracy of the decision tree. .Will As the first The voting weights of each decision tree are weighted, and the category prediction results of each decision tree are weighted to output the recognition result of the differential reflectance arc. This meta-learning-based weighted voting mechanism can dynamically adjust the contribution ratio of each decision tree, assigning higher weights to decision trees with stronger discriminative abilities, thereby improving the overall recognition accuracy. It has significant advantages, especially in scenarios where the ZDR arc region has complex morphology and uneven feature distribution.

[0068] In one optional embodiment, a meta-training sample set is formed by combining the path embedding vectors corresponding to all decision trees with the classification accuracy of each decision tree on the validation set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. The meta-random forest classifier is obtained by aggregating all meta-decision trees, including: The path embedding vectors corresponding to all decision trees are used to form a meta-feature matrix according to the decision tree index. The classification accuracy of each decision tree on the validation set is used to form a meta-label vector according to the decision tree index. The corresponding elements of each row vector of the meta-feature matrix and the meta-label vector are used to form a meta-training sample pair and a meta-training sample set. Multiple rounds of random sampling with replacement are performed on the meta-training sample set to obtain multiple meta-bootstrap sample sets. For each meta-bootstrap sample set, a corresponding meta-decision tree is constructed. The meta-training sample pairs that were not sampled in the meta-training sample set are used to form an out-of-bag sample set. The prediction error of each meta-decision tree on the corresponding out-of-bag sample set is calculated. Meta-decision trees with prediction errors lower than a preset error threshold are selected and retained. The retained meta-decision trees are aggregated to obtain a meta-random forest classifier. The voting weights of each meta-decision tree are calculated for the meta-random forest classifier. Weighted voting is performed on the prediction results of each meta-decision tree to obtain the ensemble prediction output of the meta-random forest classifier.

[0069] In one specific implementation, after training the base classifier and extracting the path embedding vectors of each decision tree, a meta-random forest classifier needs to be constructed to achieve adaptive evaluation and integration of the predictive capabilities of the base classifier. The construction of the meta-training sample set is based on the decision tree index as the alignment benchmark: the path embedding vectors corresponding to all decision trees in the base random forest are... Indexed by decision tree Arranged sequentially, forming a meta-feature matrix , of which The line is the first The path embedding vectors of each decision tree; the classification accuracy of each decision tree on the validation set. Arranged in the same index order, forming a meta-label vector. , its first The element is the th element. The accuracy value corresponding to each decision tree. The first [tree name] in the meta-feature matrix. The row vector and the meta-label vector Each element together constitutes a meta-training sample pair, and the set of all meta-training sample pairs is the meta-training sample set. .

[0070] Meta-feature matrix The number of rows is equal to the total number of decision trees in the base random forest. The number of columns equals the dimension of the path embedding vector. Since the path embedding vector encodes the topological structure of the discriminative path through a graph neural network, integrating the depth and location features of each node, the split feature index embedding, and child node aggregation information, this vector effectively represents the structural complexity and feature utilization pattern of the decision tree. The accuracy value in the meta-label vector reflects the actual discriminative ability of each decision tree on the independent validation set; the pairing of these two constitutes the supervision signal for meta-level learning.

[0071] For the meta-training sample set Perform multiple rounds of random sampling with replacement, with the number of samples sampled in each round being equal to... The total size is the same, thus resulting in multiple meta-bootstrapping sample sets. ,in This is the index for the rounds of self-bootstrapping. Since sampling is done with replacement, each meta-self-bootstrapping sample set contains approximately [number missing]. Approximately 63.2% of the original meta-training sample pairs were selected, and the remaining unselected meta-training sample pairs constituted the out-of-bag sample set for that round. For each meta-self-service sample set Construct a meta-decision tree independently. When splitting nodes in the meta-decision tree, nodes are randomly selected from all dimensions of the path embedding vector. A number of candidate dimensions are used to reduce the correlation between meta-decision trees. Each meta-decision tree recursively selects the optimal splitting dimension and splitting threshold based on the mean squared error or mean absolute error, until a preset stopping condition is met (such as the number of node samples being lower than the minimum threshold or the tree depth reaching the upper limit).

[0072] During the out-of-bag error assessment phase, the out-of-bag sample set... The path embedding vectors of each training sample pair are input into the corresponding decision tree to obtain the predicted accuracy of each decision tree, which is then compared with the true accuracy. Compare and calculate the prediction error Prediction error The mean squared error method can be used, which is the average of the squared prediction errors of all samples in the out-of-bag sample set. Prediction errors are then filtered out and retained. Below the preset error threshold The meta-decision trees are filtered out, and those that perform poorly in out-of-bag validation are removed, thereby improving the overall predictive reliability of the meta-random forest classifier. This filtering mechanism can adaptively remove meta-decision trees that are overfitted or underfitted due to bootstrap sample distribution bias, while retaining components with strong generalization ability in the structural embedding space.

[0073] After aggregating all retained meta-decision trees to form a meta-random forest classifier, it is necessary to further calculate the voting weights of each meta-decision tree to achieve weighted ensemble. The calculation of voting weights is based on the prediction accuracy of each meta-decision tree on the corresponding out-of-bag sample set: for the retained... The out-of-bag prediction error of a single-element decision tree. The lower the value, the greater the corresponding voting weight. The higher the value, the better. Specifically, the higher the voting weight. With prediction error The inverse relationship can be obtained by normalizing the inverse of the error of all retained meta-decision trees, i.e. ,in To prevent the numerical smoothing constant from being divided by zero, Traverse all retained meta-decision tree indices.

[0074] When predicting the accuracy of the decision tree to be identified, the path embedding vector of the target decision tree is input into all the retained meta-decision trees to obtain the independent prediction results of each meta-decision tree. Then according to voting weight A weighted average is applied to each prediction result to obtain the ensemble prediction output of the meta-random forest classifier. ,Right now The ensemble prediction output is the meta-random forest classifier for the first... The estimated discriminative power of each basic decision tree is used to weight the credibility of the output results of each basic decision tree in the subsequent ZDR arc recognition inference stage.

[0075] The construction of the aforementioned meta-training sample set and the training process of the meta-random forest have several key engineering implementation points. Regarding the size of the meta-feature matrix, when the number of decision trees in the basic random forest is small, the number of meta-training samples is limited. The randomness of bootstrapping may lead to an excessively small out-of-bag sample set. In this case, the size can be appropriately reduced. The rigor of the decision tree should be carefully controlled to avoid over-selection. Regarding the depth control of the meta-decision tree, since the number of meta-training samples is usually much smaller than the number of basic training samples, the maximum depth of the meta-decision tree should be set to a small value to prevent overfitting on a limited number of samples. For the dynamic updating of voting weights, if new decision tree sample pairs are continuously accumulated in practical applications, the meta-random forest classifier can be locally updated incrementally. Only the newly added samples are subjected to bootstrapping to supplement the new meta-decision tree, without retraining the existing meta-decision tree, thus reducing computational overhead.

[0076] Through the above-described construction process of the meta-random forest classifier, the automatic evaluation and differentiated utilization of the discriminative ability of each decision tree in the basic random forest are realized. This allows decision trees with stronger discriminative ability to receive higher weight contributions during the ZDR arc recognition inference stage, effectively improving the overall recognition accuracy and robustness.

[0077] A second aspect of this invention provides a random forest modeling system for automatic ZDR arc identification, comprising: The data acquisition unit is used to acquire dual-polarization radar observation data and extract the differential reflectivity field and reflectivity field. The region identification unit is used to determine the arc-shaped candidate region based on the spatial relationship between the differential reflectance field and the reflectance field, and to extract the corresponding multidimensional feature parameters. The feature extraction unit is used to calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, filter feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. A sample construction unit is used to construct a training sample set by combining the multidimensional feature parameters with higher-order interaction features; The classifier building unit is used to initialize the basic random forest and build a decision tree, calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the growth of the decision tree and obtain the basic classifier. The feature encoding unit is used to extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The result output unit is used to input the features of the data to be identified into the base classifier to obtain the discrimination path sequence, input the path embedding vector corresponding to the discrimination path sequence into the meta-random forest classifier, and output the differential reflectance arc recognition result.

[0078] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0079] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0080] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A random forest modeling method with automatic ZDR arc identification, characterized in that, include: Acquire dual-polarization radar observation data and extract differential reflectivity field and reflectivity field; Based on the spatial relationship between the differential reflectivity field and the reflectivity field, the arc-shaped candidate region is determined, and the corresponding multidimensional feature parameters are extracted. Calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, select feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. The multidimensional feature parameters and higher-order interaction features are used to construct a training sample set; Initialize the basic random forest and construct the decision tree. Calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the decision tree growth and obtain the basic classifier. Extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The features of the data to be identified are input into the base classifier to obtain the discrimination path sequence. The path embedding vector corresponding to the discrimination path sequence is input into the meta-random forest classifier, and the differential reflectance arc recognition result is output.

2. The method according to claim 1, characterized in that, The process involves calculating the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, selecting feature pairs that simultaneously meet preset threshold conditions, and performing tensor product operations to flatten and generate higher-order interactive features, including: Select any two different features from the multidimensional feature parameters to form a feature pair, calculate the joint probability distribution and marginal probability distribution of the feature pair, calculate the mutual information value based on the joint probability distribution and marginal probability distribution, and organize the mutual information values ​​of all feature pairs in the multidimensional feature parameters into a mutual information matrix according to the row and column indices. Linear regression is performed on the first feature of the feature pair to obtain the remaining features, and linear regression is performed on the second feature of the feature pair to obtain the remaining features, and the correlation coefficient between the first residual vector and the second residual vector is calculated to determine the partial correlation coefficient. The partial correlation coefficients of all feature pairs in the multidimensional feature parameters are organized into a partial correlation coefficient matrix according to the row and column indices. Traverse each position of the mutual information matrix and the partial correlation coefficient matrix. When the mutual information value at that position is greater than the preset mutual information threshold and the absolute value of the partial correlation coefficient at that position is greater than the preset partial correlation threshold, extract the feature pair corresponding to that position and form a set of filtered feature pairs from the extracted feature pairs. For each feature pair in the set of filtered feature pairs, extract the corresponding two feature vectors, perform tensor product operation to generate a higher-order interaction feature tensor, and flatten the higher-order interaction feature tensor into a one-dimensional vector to form a higher-order interaction feature.

3. The method according to claim 2, characterized in that, Select any two distinct features from the multidimensional feature parameters to form a feature pair, calculate the joint probability distribution and marginal probability distribution of the feature pair, calculate the mutual information value based on the joint probability distribution and marginal probability distribution, and organize the mutual information values ​​of all feature pairs in the multidimensional feature parameters into a mutual information matrix according to row and column indices, including: Traverse any two different features in the multidimensional feature parameters to form a feature pair. Calculate the interquartile range for the first and second features in the feature pair. Determine the adaptive binning width based on the interquartile range and the number of samples. Generate the first feature binning boundary sequence and the second feature binning boundary sequence based on the adaptive binning width. A two-dimensional binning grid is constructed based on the first feature binning boundary sequence and the second feature binning boundary sequence. Sample points are mapped to grid cells, and a two-dimensional frequency matrix is ​​constructed by statistically analyzing the frequencies. The joint probability distribution matrix is ​​then obtained by normalization. The first feature marginal probability vector is obtained by summing the joint probability distribution matrix along the row direction, and the second feature marginal probability vector is obtained by summing the joint probability distribution matrix along the column direction. Traverse the elements greater than zero in the joint probability distribution matrix, extract the corresponding index positions, read the marginal probability values ​​of the corresponding index positions from the two marginal probability vectors, and calculate the product to determine the joint probability of the independent hypothesis. Calculate the log ratio based on the elements of the joint probability distribution matrix and the joint probability of the independent hypothesis. Weight the log ratio with the elements of the joint probability distribution matrix to obtain the mutual information contribution of the grid cells. Accumulate the mutual information contributions of all grid cells to obtain the mutual information value of the feature pair. The mutual information matrix is ​​constructed by filling all feature pairs with their mutual information values ​​according to their row and column indices.

4. The method according to claim 1, characterized in that, Initialize a basic random forest and construct a decision tree. Calculate the feature conditional entropy of samples at each node of the decision tree, reconstruct the sampling probability distribution of each feature, and select splitting features to complete the decision tree growth, resulting in a basic classifier including: Initialize the basic random forest and construct a decision tree. Traverse each node of the decision tree. For the sample set of the current node, extract all values ​​of each candidate feature to form a value set. For each value in the value set, select the corresponding sample subset. Count the number of samples of each category in the sample subset to construct a category frequency vector. Calculate the information entropy based on the category frequency vector to obtain the conditional entropy corresponding to the value. Sum the conditional entropies of all values ​​of the candidate feature according to the sample subset size to obtain the feature conditional entropy. The reciprocal of the feature conditional entropy of all candidate features is taken element by element to obtain the entropy reciprocal vector. The entropy reciprocal vector is converted into a probability distribution to obtain the sampling probability vector as the sampling probability distribution of each feature. A dynamic sampling probability vector is constructed based on the sampling probability distribution and the historical splitting frequency. Weighted random sampling is performed on the candidate features to obtain the initial selection features. The splitting features are obtained based on the initial selection features. Based on the optimal split point of the splitting feature, the current node sample set is split to generate left and right child nodes. The node splitting and feature sampling are recursively performed until the preset maximum depth or preset minimum number of samples is met, and the decision tree growth is completed to obtain the basic classifier.

5. The method according to claim 4, characterized in that, A dynamic sampling probability vector is constructed based on the sampling probability distribution and historical splitting frequency. Weighted random sampling is performed on the candidate features to obtain initial selected features. Splitting features are obtained based on the initial selected features, including: Extract the sampling probability value corresponding to each candidate feature from the sampling probability vector, calculate the ratio of the sampling probability value to the historical split frequency for each candidate feature to obtain the adjustment factor, multiply the sampling probability value by the adjustment factor to determine the dynamic sampling probability, and construct a dynamic sampling probability vector for the dynamic sampling probabilities of all candidate features. Calculate the cumulative sum of the dynamic sampling probability vector to construct the cumulative probability vector, generate random numbers that follow a uniform distribution, find the position index of the first position greater than the random number in the cumulative probability vector, and extract the candidate features corresponding to the position index as the initial selection features. Calculate the information gain of the initial selected feature on the current node sample set. If the information gain is greater than the preset information gain threshold, the initial selected feature is determined as the splitting feature. Otherwise, the initial selected feature is removed from the candidate features and dynamic sampling is re-executed until the splitting feature is obtained. The historical splitting frequency of the splitting characteristics is incremented and the historical splitting frequency record is updated.

6. The method according to claim 1, characterized in that, Extracting the discriminative path sequences from each decision tree in the base classifier, converting the path sequences into a topological graph and encoding them as path embedding vectors, and training a meta-random forest classifier based on the path embedding vectors includes: Traverse each decision tree in the base classifier from the root node to the leaf node, extract the node identifiers and split features passed during the traversal to form a discrimination path sequence, map the node identifiers and split features in the discrimination path sequence to node attributes, map the parent-child relationship of adjacent nodes to directed edges, and construct the topology graph of the discrimination path based on the node attributes and directed edges. Initialize the node feature vector for each node in the topology graph. Use a message passing mechanism to perform feature aggregation layer by layer from the leaf node to the root node. For the current node, collect the node feature vectors of its child nodes and perform weighted summation to obtain the aggregated feature vector. Concatenate the aggregated feature vector with the depth position feature and split feature index of the current node to obtain the updated node feature vector. Extract the node feature vector of the root node as the path embedding vector. The path embedding vectors corresponding to all decision trees and the classification accuracy of each decision tree on the validation set are combined into a meta-training sample set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. All meta-decision trees are aggregated to obtain the meta-random forest classifier.

7. The method according to claim 1, characterized in that, The path embedding vectors corresponding to all decision trees and the classification accuracy of each decision tree on the validation set are combined into a meta-training sample set according to the decision tree index. Bootstrap sampling is performed on the meta-training sample set to construct multiple meta-decision trees. All meta-decision trees are aggregated to obtain a meta-random forest classifier, including: The path embedding vectors corresponding to all decision trees are used to form a meta-feature matrix according to the decision tree index. The classification accuracy of each decision tree on the validation set is used to form a meta-label vector according to the decision tree index. The corresponding elements of each row vector of the meta-feature matrix and the meta-label vector are used to form a meta-training sample pair and a meta-training sample set. Multiple rounds of random sampling with replacement are performed on the meta-training sample set to obtain multiple meta-bootstrap sample sets. For each meta-bootstrap sample set, a corresponding meta-decision tree is constructed. The meta-training sample pairs that were not sampled in the meta-training sample set are used to form an out-of-bag sample set. The prediction error of each meta-decision tree on the corresponding out-of-bag sample set is calculated. Meta-decision trees with prediction errors lower than a preset error threshold are selected and retained. The retained meta-decision trees are aggregated to obtain a meta-random forest classifier. The voting weights of each meta-decision tree are calculated for the meta-random forest classifier. Weighted voting is performed on the prediction results of each meta-decision tree to obtain the ensemble prediction output of the meta-random forest classifier.

8. A random forest modeling system for automatic ZDR arc identification, used to implement the method as described in any one of claims 1-7, characterized in that, include: The data acquisition unit is used to acquire dual-polarization radar observation data and extract the differential reflectivity field and reflectivity field. The region identification unit is used to determine the arc-shaped candidate region based on the spatial relationship between the differential reflectance field and the reflectance field, and to extract the corresponding multidimensional feature parameters. The feature extraction unit is used to calculate the mutual information matrix and partial correlation coefficient matrix of multidimensional feature parameters, filter feature pairs that simultaneously meet the preset threshold conditions, and perform tensor product operation to flatten and generate higher-order interactive features. A sample construction unit is used to construct a training sample set by combining the multidimensional feature parameters with higher-order interaction features; The classifier building unit is used to initialize the basic random forest and build a decision tree, calculate the feature conditional entropy of each node sample in the decision tree, reconstruct the sampling probability distribution of each feature, and select split features to complete the growth of the decision tree and obtain the basic classifier. The feature encoding unit is used to extract the discrimination path sequence of each decision tree in the base classifier, convert the path sequence into a topological graph and encode it into a path embedding vector, and train a meta-random forest classifier based on the path embedding vector; The result output unit is used to input the features of the data to be identified into the base classifier to obtain the discrimination path sequence, input the path embedding vector corresponding to the discrimination path sequence into the meta-random forest classifier, and output the differential reflectance arc recognition result.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images