An artificial intelligence-based big data mining method, system, device and medium

Abstract

The present application relates to the technical field of data processing, and particularly relates to a big data mining method, system, device and medium based on artificial intelligence, obtaining multi-dimensional heterogeneous structured data, mapping each data field in the structured data to a graph node; through a dynamic edge weight learning network based on an attention mechanism, the present application breaks the traditional split architecture, embeds a latent space constraint term in the mapping network, so that the model directly completes low-dimensional nonlinear mapping while mining features, eliminates the huge computing power overhead of calling an independent dimension reduction algorithm, and reduces the rendering delay of front-end visualization. A double-channel network with a Laplacian correction term is used to deeply mine global features, and a topological distance preserving loss function is used to impose a distance penalty gradient, which forces the associated data to naturally form highly cohesive visual clusters in the low-dimensional visualization space, avoiding the loss of original topological features in dimension reduction and improving the intuitiveness of the visualization result.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, system, device and medium for big data mining based on artificial intelligence. Background Technology

[0002] With the development of artificial intelligence and big data technologies, the mining and visualization analysis of structured data are playing a role in fields such as the Industrial Internet of Things and financial risk control. Currently, the mining and visualization processes for structured data, such as multi-dimensional sensor logs from equipment and relational transaction records, generally adopt a "two-stage independent serial" system architecture.

[0003] In the data mining and feature extraction stages, existing technologies typically lack a preprocessing step to construct an explicit topological network. Instead, they directly input heterogeneous structured data containing complex physical relationships into networks such as Long Short-Term Memory (LSTM) or conventional convolutional neural networks for computation to obtain high-dimensional abstract feature vectors. In the front-end visualization stage, since the front-end interface can only render two-dimensional or three-dimensional space, the system must call conventional dimensionality reduction algorithms (such as Principal Component Analysis (PCA) or t-SNE) independent of the mining network to perform secondary dimensionality reduction mapping on the mined high-dimensional features.

[0004] This architecture has two drawbacks. On the one hand, when faced with massive amounts of data nodes, the time complexity of distance matrix calculation by independent dimensionality reduction algorithms such as PCA or t-SNE increases exponentially, consuming system computing power and causing computational delays when processing large-scale dynamic data, making it impossible to achieve real-time rendering and updating of visualization charts. On the other hand, because the dimensionality reduction module is decoupled from the feature extraction module, the independent dimensionality reduction process lacks guidance from high-order semantic features and the global network structure. This leads to the final visualization data mapped to the low-dimensional space easily losing the features of the high-order nonlinear graph network structure, exhibiting a scattered distribution and poor cohesion, thus weakening the intuitiveness of the data mining results. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a big data mining method, system, device, and medium based on artificial intelligence, which solves the problem that the existing data mining feature extraction network and visualization dimensionality reduction module are architecturally isolated and calculated independently in sequence, resulting in high computational latency when performing secondary dimensionality reduction visualization on structured big data, and easily causing loss of structural features of high-order nonlinear graph networks.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: Step S1: Obtain multidimensional heterogeneous structured data, and map the multidimensional data record entities in the structured data as graph nodes; Step S2: Calculate the association strength between the graph nodes through a dynamic edge weight learning network based on an attention mechanism, and filter out weak association edges by using a sparse attention activation function mechanism that preserves the back gradient, thereby constructing a high-dimensional sparse topology graph. Step S3: Synchronously input the high-dimensional sparse topology graph into the preset dual-channel feature mining network; The first channel uses a one-dimensional temporal convolutional network to extract local fluctuation features of continuous time series numerical data carrying physical sensing devices in the graph nodes. Discrete categorical data is converted into dense vectors through an embedding layer and then bypasses the one-dimensional temporal convolutional network. The second channel uses a graph convolutional network with a self-loop-enhanced normalized adjacency matrix to extract global topological features of the high-dimensional sparse topological graph. Step S4: Using an adaptive fusion layer based on information entropy, feature variance and structural information entropy are used as inputs to a fully connected network. An activation function is used to output a continuous range of fusion dynamic weight tensors. The fusion weights of the local fluctuation features and the global topological features are calculated. The first channel output, the categorical data dense vector, and the second channel output are concatenated according to the weights to output the fused high-dimensional structural feature vector. Step S5: Input the high-dimensional structural feature vector into a mapping network containing a decoder and an embedded variational autoencoder latent space constraint term. Based on the preset topological distance preservation loss function and the objective function jointly optimized by data reconstruction error and probability distribution divergence, the high-dimensional structural feature vector is directly mapped to a low-dimensional visualization space of preset dimension, generating and outputting structured visualization data with clustering attributes.

[0008] As a preferred embodiment of the big data mining method based on artificial intelligence described in this invention, the step S2 of mapping each data field in the structured data to graph nodes includes: Extract the initial feature vector between any two graph nodes; The initial feature vector is input into a multi-head self-attention model to calculate the attention score between the graph nodes.

[0009] As a preferred embodiment of the big data mining method based on artificial intelligence described in this invention, step S3, which involves calculating the association strength between graph nodes using a dynamic edge weight learning network based on an attention mechanism, includes: The attention score is associated with a prior matrix of a preset physical field and then subjected to a Hadamard product operation to obtain the dynamic edge weight; The adjacency matrix of the high-dimensional sparse topology graph is updated based on the dynamic edge weights.

[0010] As a preferred embodiment of the big data mining method based on artificial intelligence described in this invention, the feature transfer rule of the graph convolutional network with self-loop enhanced normalized adjacency matrix satisfies the following expression for the self-loop enhanced normalized adjacency matrix:

[0011] in, This is the corrected Laplace matrix. Let A be the identity matrix and A be the adjacency matrix of the high-dimensional sparse topological graph. Let be the degree matrix of the high-dimensional sparse topological graph. To aggregate weight coefficients based on neighbor features that exhibit discrete exponential reduction as the number of network layers increases. This indicates taking the negative 1 / 2 power of the degree matrix.

[0012] As a preferred embodiment of the big data mining method based on artificial intelligence described in this invention, step S4, which involves calculating the input of the fully connected network through an adaptive fusion layer based on information entropy, includes: Calculate the feature variance of the local fluctuation feature and the structural information entropy of the global topological feature, respectively.

[0013] As a preferred embodiment of the artificial intelligence-based big data mining method of the present invention, the step of outputting fused features in step S4 includes: The feature variance and the structural information entropy are used as inputs to a fully connected network. The Softmax activation function is used to output a fusion dynamic weight tensor of the continuous range of the first channel features and the second channel features. The first channel output features and the second channel output features are weighted and concatenated according to the fusion dynamic weight tensor, and then concatenated with the categorical data dense vector to generate the high-dimensional structural feature vector.

[0014] As a preferred embodiment of the artificial intelligence-based big data mining method of the present invention, the mapping step in step S5 includes: Under the latent space constraints of the variational autoencoder containing the decoder, the geodesic shortest path distance between any two nodes in the high-dimensional space before dimensionality reduction is calculated based on the connection of the high-dimensional sparse graph, and the Euclidean distance between the corresponding nodes in the low-dimensional visualization space after dimensionality reduction is calculated, and the above two distances are scaled and normalized. When the normalized geodesic shortest path distance is less than the clustering decision threshold and the normalized Euclidean distance is greater than the dissipation threshold, a topological clustering distance penalty gradient is applied to the mapping network during the model training phase; when the normalized geodesic shortest path distance is greater than the preset repulsion threshold and the normalized Euclidean distance is less than the clustering threshold, a topological repulsion penalty gradient is applied to the mapping network during the model training phase, forcing the associated structured data to form cohesive visualization clusters in the low-dimensional visualization space.

[0015] Secondly, this invention provides an artificial intelligence-based big data mining system, comprising: an adaptive graph construction module, used to acquire multidimensional heterogeneous structured data, map each data field in the structured data to graph nodes, calculate the correlation strength between the graph nodes through a dynamic edge weight learning network based on an attention mechanism, and set a sparsity threshold to filter out weakly correlated edges, thereby constructing a high-dimensional sparse topology graph; a dual-channel mining module, used to synchronously input the high-dimensional sparse topology graph into a preset dual-channel feature mining network, extract features using a one-dimensional temporal convolutional network and a graph convolutional network with a Laplacian matrix correction term, and output the fused high-dimensional structural feature vector through an adaptive fusion layer based on information entropy; and a dimensionality reduction and visualization mapping module, used to input the high-dimensional structural feature vector into a mapping network with a latent space constraint term embedded in a variational autoencoder, map it to a low-dimensional visualization space of a preset dimension based on a preset topological distance preservation loss function, and generate and output structured visualization data with clustering attributes.

[0016] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of a big data mining method based on artificial intelligence.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the artificial intelligence-based big data mining method.

[0018] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention breaks away from the traditional fragmented architecture. By embedding latent space constraints within the mapping network, the model directly completes low-dimensional nonlinear mapping while mining features, eliminating the polynomial computational overhead of calling independent dimensionality reduction algorithms and reducing rendering latency in front-end visualization. It utilizes a dual-channel network employing a self-loop-enhanced normalized adjacency matrix to mine global features, and combines this with a topological distance-preserving loss function to apply bidirectional distance penalty gradients during training. This mechanism guides associated data to naturally form highly cohesive visual clusters in the low-dimensional visualization space, avoiding the loss of original topological features during dimensionality reduction and improving the intuitiveness of the visualization results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall process of a big data mining method based on artificial intelligence according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the dual-channel mining module and adaptive fusion layer of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0023] Example 1, referring to Figure 1 As an embodiment of the present invention, an overall process for a big data mining method based on artificial intelligence is provided, including: Obtain multidimensional heterogeneous structured data and map each data field in the structured data to a graph node; By using a dynamic edge weight learning network based on an attention mechanism, the association strength between the graph nodes is calculated, and a sparsity threshold is set to filter out weakly associated edges, thereby constructing a high-dimensional sparse topology graph. The high-dimensional sparse topology graph is synchronously input into a preset dual-channel feature mining network. The first channel uses a one-dimensional temporal convolutional network to extract the local fluctuation features of the numerical data in the graph nodes, and the second channel uses a graph convolutional network with a Laplacian matrix correction term to extract the global topological features of the high-dimensional sparse topology graph. The weight distribution of the local fluctuation features and the global topological features is calculated through an adaptive fusion layer based on information entropy, and the fused high-dimensional structural feature vector is output. The high-dimensional structural feature vector is input into a mapping network with a latent space constraint term embedded in a variational autoencoder. Based on a preset topological distance preserving loss function, the high-dimensional structural feature vector is directly mapped to a low-dimensional visualization space of a preset dimension, generating and outputting structured visualization data with clustering attributes.

[0024] Example 2, refer to Figure 1 This is one embodiment of the present invention. Based on the above embodiment, an artificial intelligence-based big data mining method is provided. In this embodiment, the algorithm logic, data tensor transformation process, and hyperparameter configuration of each of the above steps will be described in detail.

[0025] In this application's embodiments, acquiring multidimensional heterogeneous structured data typically refers to reading records from a distributed database. This data may simultaneously contain continuous time-series numerical fields and discrete categorical fields. Before mapping to graph nodes, the system first aligns and normalizes the heterogeneous data. For numerical data, Z-Score normalization is used, and for categorical data, one-hot encoding or an embedding layer is used to convert it into a dense vector.

[0026] The structured data is mapped to graph nodes, and the association strength between the graph nodes is calculated using a dynamic edge weight learning network based on an attention mechanism. Specifically, this includes the following steps: A1: Assume there are N nodes in the graph. Here, nodes are defined as multidimensional data record entities or key attribute fields, and the initial feature dimension of each node is Din.

[0027] A2: The initial feature vector is input into the multi-head self-attention model and projected into query, key, and value vectors. The inner product of the query vector and the key vector is calculated and divided by the square root of the dimension scaling factor to prevent gradient vanishing. The normalized attention score matrix is ​​then calculated using the Softmax activation function.

[0028] A3: The attention score is multiplied by the Hadamard product with the preset physical association prior matrix to obtain the dynamic edge weights. The system uses a sparse attention activation function mechanism that preserves the back gradient (such as the Sparsemax function) to replace the hard truncation threshold, filter out weak association edges and construct a high-dimensional sparse topology graph to ensure the differentiability of the network training end to end.

[0029] In one alternative implementation, the construction of a high-dimensional sparse topological graph can also be achieved by using principal component analysis to perform initial dimensionality screening, pruning data fields with variance contribution rates lower than a preset percentage, thereby reducing the initial dimension of the input feature matrix; and using the Pearson correlation coefficient matrix to construct a static connection graph between nodes.

[0030] In another alternative implementation, for large-scale graphs, constructing a high-dimensional sparse topological graph can also be achieved using a locality-sensitive hashing algorithm. This algorithm maps similar structured data records to the same hash bucket using a hash function, and uses the centroids of each hash bucket as supernodes to build the topological network. This approach reduces the overall computational scale of the graph network.

[0031] In this embodiment, the one-dimensional temporal convolutional network of the first channel operates on the continuous time-series data of the physical sensing device carried by the node, capturing the temporal fluctuations of the local window. Discrete categorical data is transformed into dense vectors by the embedding layer, bypassing the one-dimensional temporal convolutional network stage and directly retained to the end of the channel for fusion.

[0032] In the information entropy adaptive fusion step: B1: Calculate the characteristic variance of local fluctuation features and the structural information entropy of global topological features.

[0033] B2: The feature variance and structural information entropy are concatenated into an input vector, which is then fed into a fully connected network. The Softmax activation function is used to output a continuous range of fused dynamic weight tensors.

[0034] B3: Based on the generated continuous dynamic weights, the output features of the first channel and the output features of the second channel are weighted and then concatenated with the dense vector of categorical data from the bypass to generate a high-dimensional structural feature vector.

[0035] The dimensionality reduction and visualization mapping steps include the following steps C1 to C2: C1: The mapping network consists of an encoder and a decoder of a variational autoencoder. The system performs reparameterized sampling in the latent space and optimizes the objective function, which includes the data reconstruction error and the KL divergence of the probability distribution. The system calculates the geodesic shortest path distance between any two nodes in the high-dimensional space before dimensionality reduction, based on sparse graph connectivity, and the Euclidean distance between the corresponding nodes in the low-dimensional visualization space after dimensionality reduction, and performs scale normalization.

[0036] C2: The system employs a bidirectional penalty mechanism. When the normalized geodesic shortest path distance is less than the clustering threshold and the Euclidean distance is greater than the dissipation threshold, a topological clustering penalty gradient is applied to the mapping network during model training. Conversely, when the geodesic shortest path distance is greater than a preset repulsion threshold and the Euclidean distance is less than the clustering threshold, a topological repulsion penalty gradient is applied. This bidirectional constraint guides parameter updates during training, forcing the visualized data nodes generated during inference to form naturally cohesive clusters with clear repulsion boundaries.

[0037] Training details and hyperparameter configuration The dual-channel 1D-CNN network contains three dilated convolutional layers with kernel sizes of 3, 5, and 7. The graph convolutional network consists of four layers. The fully connected network uses LeakvReLU as its activation function. The AdamW optimizer is used, with an initial learning rate of 1e-3, and a cosine annealing strategy is employed for learning rate decay. During training, early stopping is implemented by monitoring the reconstruction loss on the validation set to prevent overfitting.

[0038] Experimental Results and Comparative Verification To verify the effectiveness of the proposed method, experiments were conducted using publicly available multidimensional sensor datasets, and the results were compared with baseline methods. Metrics included normalized mutual information, adjusted RAND index, and silhouette coefficient.

[0039]

[0040] Data shows that this solution improves the cluster compactness index, and by eliminating the need for independent distance matrix calculation, the visualization rendering latency is reduced to 23ms, supporting real-time data stream rendering.

[0041] In another alternative implementation, the dimensionality reduction mapping module can also be optimized by combining a variational autoencoder with a uniform manifold approximation and a projection loss function, thereby achieving a topological homeomorphic nonlinear mapping of features by minimizing the fuzzy set cross-entropy between the fuzzy simplex set constructed in the high-dimensional space and the simplex set in the low-dimensional space.

[0042] The method provided in this application alters the serial architecture between data mining and the front-end visualization system by constructing a dynamic topology graph, decoupling and mining dual-channel network features, and using a nonlinear dimensionality reduction mechanism with embedded potential spatial constraints, while preserving the physical topological association attributes between data fields.

[0043] Example 3 In wind farm monitoring, the system maps time slices from multiple wind turbines' common-source sensors (e.g., multi-time-step sampling records from independent vibration sensors) as graph nodes, and uses a dynamic edge-weighted network to discover the physical linkage state of multiple wind turbines under the airflow wake effect. A 1D-CNN is used to process the temporal fluctuations of the vibration sequence, and the graph network extracts the topological stress transmission relationship between the wind turbines. Finally, on the monitoring screen, the states of wind turbines that resonate at the same frequency are mapped as cohesive visual clusters.

[0044] In banking operations, the system maps various account holders as graph nodes. After filtering basic transaction data, the model extracts the topological structure of fund flow paths and renders suspicious account clusters with high-frequency loops as independent clusters on a two-dimensional screen.

[0045] Example 4 The above is an illustrative scheme for a big data mining method based on artificial intelligence. The technical solution of this system belongs to the same concept as the technical solution of the above method. This embodiment also provides a big data mining system based on artificial intelligence, which is deployed using a distributed microservice architecture, including: The adaptive graph construction module is used to obtain multidimensional heterogeneous structured data in a distributed message queue, map data fields to graph nodes, learn the network correlation strength through dynamic edge weight learning via attention mechanism, set a sparsity threshold to filter out weak correlation edges, and update and construct a high-dimensional sparse topology graph. The dual-channel mining module is used to synchronously input a high-dimensional sparse topological graph into a dual-channel feature mining network. It uses a one-dimensional temporal convolutional network and a graph convolutional network with a Laplacian matrix correction term to extract features. The fused high-dimensional structural feature vector is output through an information entropy adaptive fusion layer. The dimensionality reduction and visualization mapping module is used to input high-dimensional structural feature vectors into a mapping network with embedded variational autoencoder latent space constraints, and map them to a low-dimensional visualization space based on the topological distance preserving loss function, generating and outputting structured visualization data with clustering attributes.

[0046] This embodiment also provides an electronic device suitable for big data mining based on artificial intelligence, including a memory, a multi-core central processing unit, and a graphics processing unit. The memory stores computer-executable instructions, and the processor executes the computer-executable instructions to implement the methods proposed in the above embodiments.

[0047] This embodiment also provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the method proposed in the above embodiments. The computer software product can be stored in a computer-readable storage medium, including a computer floppy disk, read-only memory, random access memory, flash memory, solid-state drive, optical disk, or a distributed cloud storage cluster.

[0048] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0049] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A big data mining method based on artificial intelligence, characterized in that, Includes the following steps: Step S1: Obtain multidimensional heterogeneous structured data, and map the multidimensional data record entities in the structured data as graph nodes; Step S2: Calculate the association strength between the graph nodes through a dynamic edge weight learning network based on an attention mechanism, and filter out weak association edges by using a sparse attention activation function mechanism that preserves the back gradient, thereby constructing a high-dimensional sparse topology graph. Step S3: Synchronously input the high-dimensional sparse topology graph into the preset dual-channel feature mining network; The first channel uses a one-dimensional temporal convolutional network to extract local fluctuation features of continuous time series numerical data carrying physical sensing devices in the graph nodes. Discrete categorical data is converted into dense vectors through an embedding layer and then bypasses the one-dimensional temporal convolutional network. The second channel uses a graph convolutional network with a self-loop-enhanced normalized adjacency matrix to extract global topological features of the high-dimensional sparse topological graph. Step S4: Using an adaptive fusion layer based on information entropy, feature variance and structural information entropy are used as inputs to a fully connected network. An activation function is used to output a continuous range of fusion dynamic weight tensors. The fusion weights of the local fluctuation features and the global topological features are calculated. The first channel output, the categorical data dense vector, and the second channel output are concatenated according to the weights to output the fused high-dimensional structural feature vector. Step S5: Input the high-dimensional structural feature vector into a mapping network containing a decoder and an embedded variational autoencoder latent space constraint term. Based on the preset topological distance preservation loss function and the objective function jointly optimized by data reconstruction error and probability distribution divergence, the high-dimensional structural feature vector is directly mapped to a low-dimensional visualization space of preset dimension, generating and outputting structured visualization data with clustering attributes.

2. The big data mining method based on artificial intelligence as described in claim 1, characterized in that, The step S2, which maps each data field in the structured data to a graph node, includes: Extract the initial feature vector between any two graph nodes; The initial feature vector is input into a multi-head self-attention model to calculate the attention score between the graph nodes.

3. The big data mining method based on artificial intelligence as described in claim 2, characterized in that, The step S3, which involves calculating the association strength between graph nodes using a dynamic edge weight learning network based on an attention mechanism, includes: The attention score is associated with a prior matrix of a preset physical field and a Hadamard product is performed to obtain the dynamic edge weight. The adjacency matrix of the high-dimensional sparse topology graph is updated based on the dynamic edge weights.

4. The big data mining method based on artificial intelligence as described in claim 3, characterized in that, The feature transfer rule of the graph convolutional network that introduces a self-loop-enhanced normalized adjacency matrix satisfies the following expression for the self-loop-enhanced normalized adjacency matrix: in, This is the corrected Laplace matrix. Let A be the identity matrix and A be the adjacency matrix of the high-dimensional sparse topological graph. Let be the degree matrix of the high-dimensional sparse topological graph. To aggregate weight coefficients based on neighbor features that exhibit discrete exponential reduction as the number of network layers increases. This indicates taking the negative 1 / 2 power of the degree matrix.

5. The big data mining method based on artificial intelligence as described in claim 4, characterized in that, The step S4, which involves calculating the input to the fully connected network using an adaptive fusion layer based on information entropy, includes: Calculate the feature variance of the local fluctuation feature and the structural information entropy of the global topological feature, respectively.

6. The big data mining method based on artificial intelligence as described in claim 5, characterized in that, The step of outputting the fused features in step S4 includes: The feature variance and the structural information entropy are used as inputs to a fully connected network. The Softmax activation function is used to output a fusion dynamic weight tensor of the continuous range of the first channel features and the second channel features. The first channel output features and the second channel output features are weighted and concatenated according to the fusion dynamic weight tensor, and then concatenated with the categorical data dense vector to generate the high-dimensional structural feature vector.

7. The big data mining method based on artificial intelligence as described in claim 6, characterized in that, The mapping step in step S5 includes: Under the latent space constraints of the variational autoencoder containing the decoder, the geodesic shortest path distance between any two nodes in the high-dimensional space before dimensionality reduction is calculated based on the connection of the high-dimensional sparse graph, and the Euclidean distance between the corresponding nodes in the low-dimensional visualization space after dimensionality reduction is calculated, and the above two distances are scaled and normalized. When the normalized geodesic shortest path distance is less than the clustering decision threshold and the normalized Euclidean distance is greater than the dissipation threshold, a topological clustering distance penalty gradient is applied to the mapping network during the model training phase; when the normalized geodesic shortest path distance is greater than the preset repulsion threshold and the normalized Euclidean distance is less than the clustering threshold, a topological repulsion penalty gradient is applied to the mapping network during the model training phase, forcing the associated structured data to form cohesive visualization clusters in the low-dimensional visualization space.

8. A big data mining system based on artificial intelligence, employing the method described in any one of claims 1-7, characterized in that, include: An adaptive graph construction module is used to acquire multidimensional heterogeneous structured data, map multidimensional data record entities or key attribute fields in the structured data to graph nodes, calculate the association strength between the graph nodes through a dynamic edge weight learning network based on an attention mechanism, and filter out weak association edges by using a sparse attention activation function mechanism that preserves the inverse gradient, thereby constructing a high-dimensional sparse topology graph. The dual-channel mining module is used to synchronously input the high-dimensional sparse topology graph into a preset dual-channel feature mining network. It extracts features using a one-dimensional temporal convolutional network and a graph convolutional network with a self-loop-enhanced normalized adjacency matrix, and outputs the fused high-dimensional structural feature vector through an adaptive fusion layer based on information entropy. The dimensionality reduction and visualization mapping module is used to input the high-dimensional structural feature vector into a mapping network containing a decoder and an embedded variational autoencoder latent space constraint term. Based on a preset topological distance preserving loss function and an objective function jointly optimized by data reconstruction error and probability distribution divergence, it maps the data to a low-dimensional visualization space of a preset dimension, generating and outputting structured visualization data with clustering attributes.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the big data mining method based on artificial intelligence as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the big data mining method based on artificial intelligence as described in any one of claims 1 to 7.

Images