A large-scale data feature extraction method, system and device based on mean field

Abstract

The application relates to the technical field of mass data processing, and provides a large-scale data feature extraction method, system and equipment based on an average field, which comprises the following steps: dividing a feature correlation graph of a data set into a plurality of subgraphs, dividing the data set into a same number of subsets as the subgraphs, each subset containing all data samples and only containing features of a corresponding subgraph; sending each subset to a computing node, so that the computing node combines a coupling coefficient to calculate a probability distribution, an expectation and an entropy of each feature through a distributed average field equation; after updating the coupling coefficient based on the probability distribution, the expectation and the entropy of each feature calculated by all the computing nodes, the computing node is sent a computing instruction and the updated coupling coefficient until iteration convergence; and for each feature, a global probability distribution is calculated based on the probability distribution calculated by the computing node containing the feature. The network communication cost bottleneck of super large-scale data feature extraction is effectively broken through.

Description

Technical Field

[0001] This invention belongs to the field of massive data processing technology, and in particular relates to a method, system and device for large-scale data feature extraction based on mean field. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, the increasing prevalence of IoT devices, high-definition sensors, and social media has directly spurred an explosive growth in data volume. This data is not only massive in quantity but also often exhibits complex, high-dimensional, and unstructured characteristics, posing significant challenges to data feature extraction. Against this backdrop, how to quickly and accurately extract valuable information and patterns from massive amounts of data has become a core issue in promoting the application of artificial intelligence, optimizing business decisions, and supporting scientific research.

[0004] To address this issue, two main technical solutions have emerged:

[0005] One type is a distributed learning framework based on gradient descent, whose core is to optimize the model on a computing cluster by using the stochastic gradient descent (SGD) algorithm through a data or model parallel strategy.

[0006] Another type of approach is based on probabilistic graphical models, including Bayesian networks and Markov random fields, which are mainly used to characterize complex dependencies between data. In order to reduce the computational cost of probabilistic graphical models, mean-field variational inference has become a commonly used approximation method, which simplifies the process of solving the posterior distribution by assuming that the variables are independent.

[0007] However, it should be noted that existing technologies still have inherent limitations when dealing with truly massive data tasks, and cannot meet the requirements for efficient processing. From a practical application perspective, existing technologies mainly face three bottlenecks:

[0008] First, there is the communication bottleneck problem: In the distributed framework based on SGD, worker nodes need to frequently exchange gradient information with the parameter server to achieve model synchronization. As the number of model parameters increases or the cluster size expands, this process will generate huge network communication overhead, often making the communication time far exceed the computation time, resulting in a sharp decline in the scalability of the entire system, and even the paradox of performance degradation as the scale increases.

[0009] Secondly, traditional methods are generally constrained by computational complexity: many traditional mean-field variational inference methods require global computation on the entire dataset, resulting in computational complexity that is proportional to the square of the data volume or feature dimension, which severely limits their applicability in ultra-large-scale scenarios. The root of this problem is that traditional methods lack a mechanism to effectively decompose high-dimensional complex problems into low-dimensional subproblems.

[0010] In addition, insufficient flexibility and adaptability are also obvious shortcomings of existing technologies: in many existing solutions, key parameters are often statically preset or rely on costly global statistical calculations, making it difficult to adapt to dynamic changes in data distribution. This rigid parameter setting method stems from the failure of algorithm design to make full use of real-time feedback information generated during local calculations. Summary of the Invention

[0011] To address the technical problems mentioned above, this invention provides a method, system, and device for large-scale data feature extraction based on mean field. It intelligently partitions large-scale data through feature correlation analysis, decoupling the complex global solution problem into multiple independent, parallel-processable local sub-problems. Furthermore, it introduces a dynamic coupling mechanism based on information entropy and variance to coordinate the solution process of each local sub-problem, achieving rapid convergence to the global optimal solution with minimal communication overhead. This effectively overcomes the network communication overhead bottleneck in ultra-large-scale data feature extraction.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] The first aspect of this invention provides a method for large-scale data feature extraction based on mean field, applied to a parameter server, comprising:

[0014] Obtain the dataset, divide the feature association graph of the dataset into several subgraphs, divide the dataset into the same number of subsets as the subgraphs, each subset contains all data samples and only contains the features of the corresponding subgraph;

[0015] Each subset is sent to a computing node, which then uses the coupling coefficients to compute the probability distribution, expectation, and entropy of each feature through the distributed mean-field equation.

[0016] Based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes, the coupling coefficient is updated, and then the computing instructions and the updated coupling coefficient are sent to the computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0017] Furthermore, the step of updating the coupling coefficient includes: for each feature, calculating the global entropy based on the entropy calculated by the computing node containing the feature; calculating the global expectation based on the expectation calculated by the computing node containing the feature; calculating the variance based on the global expectation and the expectation calculated by the computing node containing the feature; and updating the coupling coefficient based on the global entropy and the variance through an entropy-variance weighting mechanism.

[0018] Furthermore, the entropy-variance weighting mechanism is as follows:

[0019]

[0020] in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

[0021] Furthermore, the construction steps of the feature association graph include: calculating the feature mutual information matrix based on the dataset using the mutual information formula; treating each feature in the dataset as a graph node, and constructing the feature association graph using the value of the mutual information matrix as the edge weight.

[0022] Furthermore, the condition for the iteration to converge is that the Jensen-Shannon divergence between the global feature field distribution of the current round and the previous round is less than a threshold, and the global feature field distribution is the product of the global probability distributions of all features.

[0023] Furthermore, the threshold is: ;in, As the initial threshold, The attenuation rate, This represents the number of iterations.

[0024] Furthermore, the distributed mean-field equation is:

[0025] ;

[0026] in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient.

[0027] A second aspect of the present invention provides a method for large-scale data feature extraction based on mean field, applied to computing nodes, comprising:

[0028] Receive a subset of parameters sent by the server;

[0029] In response to the computation instructions sent by the parameter server and receiving the coupling coefficients sent by the parameter server, for each feature in the subset, the probability distribution, expectation, and entropy are calculated by combining the coupling coefficients and the distributed mean field equations, and sent to the parameter server so that the parameter server updates the coupling coefficients based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0030] The parameter server divides the feature association graph of the dataset into several subgraphs, and divides the dataset into the same number of subsets as the subgraphs. Each subset contains all data samples and only contains the features of the corresponding subgraph.

[0031] A third aspect of the present invention provides a large-scale data feature extraction system based on mean field, including a parameter server and multiple computing nodes;

[0032] The parameter server is used to acquire the dataset, divide the feature association graph of the dataset into several subgraphs, divide the dataset into the same number of subsets as the subgraphs, and each subset contains all data samples and only contains the features of the corresponding subgraph.

[0033] The computing node is used to receive a subset sent by the parameter server; respond to the computing instructions sent by the parameter server and receive the coupling coefficient sent by the parameter server; for each feature in the subset, in combination with the coupling coefficient, calculate the probability distribution, expectation and entropy through the distributed mean field equation;

[0034] The parameter server is also used to update the coupling coefficients based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes, and then issue calculation instructions and updated coupling coefficients to the computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0035] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method for large-scale data feature extraction based on mean field.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention intelligently partitions large-scale data through feature association analysis, decoupling the complex global solution problem into multiple independent and parallel local subproblems. It also introduces a dynamic coupling mechanism based on information entropy and variance to coordinate the solution process of each local subproblem, achieving rapid convergence to the global optimal solution with minimal communication overhead, effectively breaking through the network communication overhead bottleneck of feature extraction for ultra-large-scale data.

[0038] By setting a dynamically changing threshold, this invention can make timely use of local real-time feedback information, adapt to dynamic changes in data distribution, and make the algorithm more flexible. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a flowchart of a large-scale data feature extraction method based on mean field according to Embodiment 1 of the present invention;

[0041] Figure 2 This is a schematic diagram of the working interaction between the parameter server cluster and each computing node in Embodiment 4 of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of a computer device according to Embodiment 5 of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0044] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0045] Example 1

[0046] This embodiment provides a method for large-scale data feature extraction based on mean field, which innovatively embeds the mean field concept into a distributed computing architecture. It can break through the performance limitations of massive data feature extraction, significantly optimize the computational bottleneck and computational complexity of feature extraction tasks, and provide technical support for massive data-driven application scenarios.

[0047] This embodiment provides a method for large-scale data feature extraction based on mean field. First, it intelligently divides the large-scale data through feature correlation analysis, decoupling the complex global solution problem into multiple local sub-problems that can be processed independently and in parallel. Second, it introduces a dynamic coupling mechanism based on information entropy and variance to coordinate the solution process of each local sub-problem. Finally, it achieves rapid convergence to reach the global optimal solution with minimal communication overhead, effectively breaking through the performance bottleneck of ultra-large-scale data feature extraction.

[0048] This embodiment provides a method for large-scale data feature extraction based on mean field, such as... Figure 1 As shown, it includes five stages, and the specific steps are as follows:

[0049] Step 1: Obtain the original large-scale data and construct a feature association graph.

[0050] To process large-scale datasets, the first step is to construct a corresponding feature association graph. The main construction steps are as follows:

[0051] (1) The large-scale data to be processed Convert to data matrix ,in Represents the number of data samples. This represents the number of data features.

[0052] Specifically, for structured data, numerical features can be directly standardized or categorical features can be one-hot encoded; for unstructured text, image, and audio data, word embedding, CNN (convolutional neural network) feature extraction, spectrogram analysis, and other methods can be used to transform them into fixed-length feature vectors.

[0053] (2) Based on data matrix Calculate the feature mutual information matrix .

[0054] Specifically, the statistical dependencies between features can be calculated using the mutual information formula, which allows us to compute the feature mutual information matrix of the data. :

[0055] ;

[0056] in, It is a random variable, representing the sum of all samples in the dataset at the _th ... The distribution of values ​​across each feature dimension; Indicates the first The marginal probability distribution of a feature, that is, the distribution pattern of the value of the feature itself; Indicates the first The first feature and the first The joint probability distribution of features, that is, the pattern of the co-occurrence of values ​​between two features; and Through data matrix Calculated.

[0057] (3) Based on the feature mutual information matrix Construct a feature association graph.

[0058] Specifically, each feature dimension Treat them as graph nodes and use the mutual information matrix The value is the edge weight, and a weighted feature association graph is constructed. .

[0059] Furthermore, in order to make the diagram Removing a large number of weak or even meaningless connections requires threshold filtering. The specific steps are as follows: set a threshold... For the graph Medium weight Remove the edges and keep the rest to form a new feature association graph. .

[0060] Furthermore, threshold The selection can be made based on the domain and characteristics of the data to be processed.

[0061] Step 2: Intelligent data segmentation.

[0062] Specifically, in order to decompose the high-dimensional feature space, graph partitioning algorithms such as spectral clustering can be used to partition the feature association graph. Divided into A tightly connected subgraph There may be feature overlap between different subgraphs, meaning that a feature may belong to two or more classes at the same time.

[0063] Furthermore, to reduce the data dimensionality that each computing node needs to process, the original dataset can be divided according to the feature partitioning results. Divided into the same number Subset And let each subset (data sub-block) Includes all data samples, but only retains those belonging to the subgraph. The feature dimensions.

[0064] Furthermore, the main purpose of this step is to make the features within each subset highly correlated, while the features between different subsets are as independent as possible. By implementing this step, high-dimensional complex problems can be decomposed into low-dimensional simple subproblems, thus significantly reducing computational complexity.

[0065] Step 3: Update the distributed mean-field equations in parallel.

[0066] Specifically, each computing node receives a subset of data. and the current global coupling coefficient Then, in a GPU-accelerated environment, each computing node independently and in parallel updates the following mean-field equation:

[0067] ;

[0068] in, Indicates the first During the round of iteration, the first Each computing node has its assigned features The best local estimate of the probability distribution; It is a temperature hyperparameter used to control the smoothness of the optimization; It is the first The normalization factor for each computing node, used to ensure... It is an effective probability distribution; Represents the current feature The set of indices of the neighbor features; It is based on the concept of mean field and represents the first... Features of all neighbors on each computing node For current features The average resultant force produced; This represents the feature obtained in the (t-1)th iteration. and neighbor characteristics The coupling coefficient between them; This indicates that the neighbor features are obtained in the (t-1)th iteration. The expectation at the k-th computation node; partial derivative Derived from the energy function right The gradient drives The distribution of the data evolves in the direction of reducing reconstruction error.

[0069] Furthermore, the purpose of this equation is to update the probability distribution of each feature dimension in each computing node. Once a probability distribution is obtained, various statistics of this feature can be easily calculated: the expectation of the calculation node. That is, the best local estimate of the feature at that node; entropy That is, the degree of disorder of the distribution at the computing node, where Represents Shannon entropy.

[0070] Furthermore, the energy function in this equation The measurement is based on features Reconstructed dataset and subset The error, for example, can be set as:

[0071]

[0072] in, Represents restructuring.

[0073] Furthermore, it should be noted that because each computing node sees data from a different perspective, different computing nodes... It may be for the same feature A slightly different distribution estimate was obtained.

[0074] Furthermore, by decoupling the complex global equation solving problem into multiple independent and parallel local average field subproblems through the above method, the network communication overhead in the data feature extraction process can be significantly reduced.

[0075] Step 4: Global coupling coefficient aggregator, update the global coupling coefficient.

[0076] (1) Calculate the global expectation.

[0077] Specifically, in each iteration, after the parameter server receives the local statistics sent by all working nodes (computing nodes), for each feature... The global expectation is calculated by averaging the local expectations of each computing node. ,in, Indicates the inclusion of features The number of computing nodes.

[0078] Furthermore, in the process of calculating the global expectation, if the size difference between data subsets is too large, the calculation can be weighted according to the size of the data subsets.

[0079] (2) Calculate the global entropy.

[0080] Specifically, for each feature Its global entropy It can be obtained from the local entropy average, that is ,in, Indicates the first During round iteration, for features The global authoritative distribution of the probability distribution can also be considered as It is in the The most likely distribution after rounds of iterations integrate all local information.

[0081] Furthermore, It is through the parameter server that all computation nodes containing this feature are associated with the feature. The global perception of a feature is formed by aggregating its statistical characteristics, i.e., the global probability distribution.

[0082] (3) Calculate the variance.

[0083] Specifically, for each feature Calculate the variance of its expected value across different computation nodes. ,Right now .

[0084] (4) Update the global dynamic coupling coefficient .

[0085] Specifically, the global coupling coefficient is updated using the following entropy-variance weighted mechanism:

[0086]

[0087] in, It is a hyperparameter used to adjust the overall importance of entropy components.

[0088] Step 5: Adaptive convergence decision maker, perform adaptive convergence decision.

[0089] This step mainly demonstrates how to intelligently determine whether the distributed mean field iteration has converged, thereby terminating the calculation and outputting the final feature extraction result. The main process is as follows:

[0090] (1) Calculate the change.

[0091] Specifically, based on the collected global field distribution information, the current iteration round is calculated. Global feature field distribution Compared to the previous round Global feature field distribution Jensen-Shannon divergence between:

[0092] ;

[0093] in, ( ) indicates the KL distance; It is the average distribution of two rounds, that is: .

[0094] Furthermore, in the above formula Indicates in All features in round iteration The joint probability distribution describes the relationships between all features and the state of the entire system.

[0095] Furthermore, according to the mean-field approximation theory, a complex joint distribution can be approximated as the product of the marginal distributions of all individuals. The update equation is:

[0096] .

[0097] Furthermore, Jensen-Shannon divergence The magnitude of the value can reflect the first The smaller the value of the overall update magnitude of the system brought about by each iteration, the more stable the system and the weaker the change. Therefore, it can be determined based on the magnitude of the update magnitude in each iteration. The magnitude of the numerical update is determined.

[0098] (2) Calculate the current threshold.

[0099] Specifically, based on the current iteration number ,calculate ,in, The initial threshold should be set to a relatively high value (e.g., 0.5). This parameter ensures that the distribution can change significantly in the early stages of iteration, encouraging the algorithm to "explore" and avoid getting trapped in local optima too early. The decay rate controls the rate at which the threshold decreases.

[0100] Furthermore, with the number of iterations The increase in threshold The threshold decreases automatically and gradually, which indicates that a "coarse-to-fine" optimization strategy has been set: the threshold is large in the early stage, allowing the algorithm to move forward in large strides and quickly approach the optimal solution region; the threshold is small in the later stage, requiring the algorithm to be refined. Iteration continues only when the distribution has improved by an extremely small amount. Therefore, it can make timely use of local real-time feedback information, adapt to the dynamic changes in data distribution, and make the algorithm more flexible.

[0101] (3) Comparison and convergence judgment.

[0102] Specifically, the following comparisons are made:

[0103] ;

[0104] As shown in the above formula, if the comparison is True, the parameter server sends a termination signal to all computing nodes and outputs the final result; otherwise, if the comparison is False, the next iteration begins.

[0105] Furthermore, after completing the iteration and convergence, its final output is a stable, global feature field distribution. Specifically, this manifests as the probability distribution and statistics of each feature. These outputs constitute a novel and efficient representation of the original dataset, which can be used for subsequent, more advanced tasks.

[0106] This embodiment provides a large-scale data feature extraction method based on mean field, which does not use the existing SGD technology. Instead, it decouples the complex global equation solving problem into multiple local mean field subproblems that can be processed independently and in parallel, thereby significantly reducing the network communication overhead bottleneck in the data feature extraction process.

[0107] This embodiment provides a method for large-scale data feature extraction based on mean field. By intelligently dividing the high-dimensional feature space, the complex high-dimensional problem is decomposed into simple low-dimensional sub-problems, thus significantly reducing computational complexity.

[0108] This embodiment provides a large-scale data feature extraction method based on mean field. By setting a dynamically changing threshold, it can make timely use of local real-time feedback information and adapt to the dynamic changes in data distribution, making the algorithm more flexible.

[0109] Example 2

[0110] This embodiment provides a large-scale data feature extraction method based on mean field, applied to a parameter server, including:

[0111] Obtain the dataset, divide the feature association graph of the dataset into several subgraphs, divide the dataset into the same number of subsets as the subgraphs, each subset contains all data samples and only contains the features of the corresponding subgraph;

[0112] Each subset is sent to a computing node, which then uses the coupling coefficients to compute the probability distribution, expectation, and entropy of each feature through the distributed mean-field equation.

[0113] Based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes, the coupling coefficient is updated, and then the computing instructions and the updated coupling coefficient are sent to the computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0114] Furthermore, the step of updating the coupling coefficient includes: for each feature, calculating the global entropy based on the entropy calculated by the computing node containing the feature; calculating the global expectation based on the expectation calculated by the computing node containing the feature; calculating the variance based on the global expectation and the expectation calculated by the computing node containing the feature; and updating the coupling coefficient based on the global entropy and the variance through an entropy-variance weighting mechanism.

[0115] Furthermore, the entropy-variance weighting mechanism is as follows:

[0116]

[0117] in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

[0118] Furthermore, the construction steps of the feature association graph include: calculating the feature mutual information matrix based on the dataset using the mutual information formula; treating each feature in the dataset as a graph node, and constructing the feature association graph using the value of the mutual information matrix as the edge weight.

[0119] Furthermore, the condition for the iteration to converge is that the Jensen-Shannon divergence between the global feature field distribution of the current round and the previous round is less than a threshold, and the global feature field distribution is the product of the global probability distributions of all features.

[0120] Furthermore, the threshold is: ;in, As the initial threshold, The attenuation rate, This represents the number of iterations.

[0121] Furthermore, the distributed mean-field equation is:

[0122] ;

[0123] in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient.

[0124] It should be noted that each step in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0125] Example 3

[0126] This embodiment provides a large-scale data feature extraction method based on mean field, applied to computing nodes, including:

[0127] Receive a subset of parameters sent by the server;

[0128] In response to the computation instructions sent by the parameter server and receiving the coupling coefficients sent by the parameter server, for each feature in the subset, the probability distribution, expectation, and entropy are calculated by combining the coupling coefficients and the distributed mean field equations, and sent to the parameter server so that the parameter server updates the coupling coefficients based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0129] The parameter server divides the feature association graph of the dataset into several subgraphs, and divides the dataset into the same number of subsets as the subgraphs. Each subset contains all data samples and only contains the features of the corresponding subgraph.

[0130] It should be noted that each step in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0131] Example 4

[0132] This embodiment provides a large-scale data feature extraction system based on mean field, such as... Figure 2 As shown, it includes a parameter server cluster (abbreviated as parameter server) and multiple computing nodes (computing node 1, computing node 2, ..., computing node K).

[0133] The parameter server is used to acquire the dataset, divide the feature association graph of the dataset into several subgraphs, and divide the dataset into subsets (data blocks) of the same number as the subgraphs. Each subset contains all data samples and only contains the features of the corresponding subgraph;

[0134] The computing node is used to receive a subset sent by the parameter server; respond to the computing instructions sent by the parameter server and receive the coupling coefficient sent by the parameter server; for each feature in the subset, in combination with the coupling coefficient, calculate the probability distribution, expectation and entropy through the distributed mean field equation;

[0135] The parameter server is also used to update the coupling coefficients based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes, and then issue calculation instructions (convergence instructions) and updated coupling coefficients to the computing nodes until iterative convergence. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature.

[0136] The parameter server includes a global coupling coefficient aggregator, which is used to update the coupling coefficients: for each feature, the global entropy is calculated based on the entropy calculated by the computing nodes containing the feature; the global expectation is calculated based on the expectation calculated by the computing nodes containing the feature; the variance is calculated based on the global expectation and the expectation calculated by the computing nodes containing the feature; and the coupling coefficients are updated based on the global entropy and variance through an entropy-variance weighting mechanism and synchronized to the adaptive convergence arbiter.

[0137] The parameter server includes an adaptive convergence determiner, which is used to determine whether the convergence condition is met. When the iteration converges, for each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature. The condition for iterative convergence is that the Jensen-Shannon divergence between the global feature field distribution of the current round and the previous round is less than a threshold. The global feature field distribution is the product of the global probability distributions of all features.

[0138] Furthermore, the entropy-variance weighting mechanism is as follows:

[0139]

[0140] in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

[0141] Furthermore, the construction steps of the feature association graph include: calculating the feature mutual information matrix based on the dataset using the mutual information formula; treating each feature in the dataset as a graph node, and constructing the feature association graph using the value of the mutual information matrix as the edge weight.

[0142] Furthermore, the condition for the iteration to converge is that the Jensen-Shannon divergence between the global feature field distribution of the current round and the previous round is less than a threshold, and the global feature field distribution is the product of the global probability distributions of all features.

[0143] Furthermore, the threshold is: ;in, As the initial threshold, The attenuation rate, This represents the number of iterations.

[0144] Furthermore, the distributed mean-field equation is:

[0145] ;

[0146] in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient.

[0147] It should be noted that each execution terminal in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0148] Example 5

[0149] This embodiment provides a computer device, such as... Figure 3As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and transmit data. When the processor 1001 executes the program, it implements the steps of the large-scale data feature extraction method based on mean field as described in Embodiment 1 above.

[0150] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for feature extraction from large-scale data based on mean field, characterized in that, Applied to parameter servers, including: Obtain the dataset, divide the feature association graph of the dataset into several subgraphs, divide the dataset into the same number of subsets as the subgraphs, each subset contains all data samples and only contains the features of the corresponding subgraph; Each subset is sent to a computing node, which then uses the coupling coefficients to compute the probability distribution, expectation, and entropy of each feature through the distributed mean-field equation. Based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes, the coupling coefficient is updated, and then the calculation instructions and the updated coupling coefficient are sent to the computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature. The distributed mean-field equation is: ； in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient; The steps for updating the coupling coefficient include: for each feature, calculating the global entropy based on the entropy calculated by the computing node containing the feature; calculating the global expectation based on the expectation calculated by the computing node containing the feature; calculating the variance based on the global expectation and the expectation calculated by the computing node containing the feature; and updating the coupling coefficient based on the global entropy and variance through an entropy-variance weighted mechanism. The entropy-variance weighting mechanism is as follows: in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

2. The method for large-scale data feature extraction based on mean field as described in claim 1, characterized in that, The steps for constructing the feature association graph include: calculating the feature mutual information matrix based on the dataset using the mutual information formula; treating each feature in the dataset as a graph node and constructing the feature association graph using the value of the mutual information matrix as the edge weight.

3. The method for large-scale data feature extraction based on mean field as described in claim 1, characterized in that, The condition for the iteration to converge is that the Jensen-Shannon divergence between the global feature field distribution of the current round and the previous round is less than a threshold, and the global feature field distribution is the product of the global probability distributions of all features.

4. The method for large-scale data feature extraction based on mean field as described in claim 3, characterized in that, The threshold is: ;in, As the initial threshold, The attenuation rate, This represents the number of iterations.

5. A method for feature extraction from large-scale data based on mean field, characterized in that, Applied to compute nodes, including: Receive a subset of parameters sent by the server; In response to the computation instructions sent by the parameter server and receiving the coupling coefficients sent by the parameter server, for each feature in the subset, the probability distribution, expectation, and entropy are calculated by combining the coupling coefficients and the distributed mean field equations, and sent to the parameter server so that the parameter server updates the coupling coefficients based on the probability distribution, expectation, and entropy of each feature calculated by all computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature. The parameter server divides the feature association graph of the dataset into several subgraphs and divides the dataset into the same number of subsets as the subgraphs. Each subset contains all data samples and only contains the features of the corresponding subgraph. The distributed mean-field equation is: ； in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient; The steps for updating the coupling coefficient include: for each feature, calculating the global entropy based on the entropy calculated by the computing node containing the feature; calculating the global expectation based on the expectation calculated by the computing node containing the feature; calculating the variance based on the global expectation and the expectation calculated by the computing node containing the feature; and updating the coupling coefficient based on the global entropy and variance through an entropy-variance weighted mechanism. The entropy-variance weighting mechanism is as follows: in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

6. A large-scale data feature extraction system based on mean field, characterized in that, Includes a parameter server and multiple computing nodes; The parameter server is used to acquire the dataset, divide the feature association graph of the dataset into several subgraphs, divide the dataset into the same number of subsets as the subgraphs, and each subset contains all data samples and only contains the features of the corresponding subgraph. The computing node is used to receive a subset sent by the parameter server; respond to the computing instructions sent by the parameter server and receive the coupling coefficient sent by the parameter server; for each feature in the subset, in combination with the coupling coefficient, calculate the probability distribution, expectation and entropy through the distributed mean field equation; The parameter server is also used to update the coupling coefficient based on the probability distribution, expectation and entropy of each feature calculated by all computing nodes, and then issue calculation instructions and updated coupling coefficients to the computing nodes until the iteration converges. For each feature, the global probability distribution is calculated based on the probability distribution calculated by the computing nodes containing that feature. The distributed mean-field equation is: ； in, Indicates the first During the round of iteration, the first Features computed by each computing node The probability distribution; It is a temperature hyperparameter; It is the first Normalization factor for each computing node; Representative characteristics The set of indices of the neighbor features; Representing the Features of all neighbors on each computing node Features The average resultant force produced; partial derivatives Energy function Features The gradient; The steps for updating the coupling coefficient include: for each feature, calculating the global entropy based on the entropy calculated by the computing node containing the feature; calculating the global expectation based on the expectation calculated by the computing node containing the feature; calculating the variance based on the global expectation and the expectation calculated by the computing node containing the feature; and updating the coupling coefficient based on the global entropy and variance through an entropy-variance weighted mechanism. The entropy-variance weighting mechanism is as follows: in, Represents the coupling coefficient obtained in the t-th iteration; Representation of features global entropy; Representation of features global entropy; Representation of features Variance between different computation nodes; Representation of features Variance between different computation nodes; It's a hyperparameter.

7. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for large-scale data feature extraction based on mean field as described in any one of claims 1-4 or 5.

Images