Big data intelligent analysis platform and method based on AI

Abstract

The invention provides an AI-based big data intelligent analysis platform and method, and relates to the field of data analysis. Target analysis original data acquired by a big data acquisition module is preprocessed, and then feature extraction and context semantic association are performed on the preprocessed data; and finally, obtaining a visual analysis result of the target analysis original data through a generator. Therefore, information in the data can be more effectively and deeply mined and utilized, potential modes and laws are captured, and finally the potential modes and laws are presented in a visual and easy-to-understand mode.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and more specifically, to an AI-based big data intelligent analysis platform and method. Background Art

[0002] Globally, with the continuous popularization and development of mobile Internet and Internet of Things technologies, the generation and accumulation of large-scale data has become a trend. However, faced with such a huge amount of data, traditional statistical methods can no longer meet the needs of data analysis and exploration. Specifically, traditional big data analysis algorithms are often based on relatively fixed patterns and assumptions. For today's complex, diverse, high-dimensional and uneven data, it is difficult to mine potential patterns and laws. For example, when the data has a lot of quality problems such as noise and missing values, and the dimension is so high that it exceeds its effective processing range, the limitations of the algorithm in accuracy and precision will become prominent, and it is difficult to accurately extract valuable information from the data.

[0003] Therefore, an AI-based big data intelligent analysis solution is needed. Summary of the invention

[0004] In view of the shortcomings of the prior art, this application provides an AI-based big data intelligent analysis platform and method, which pre-processes the target analysis raw data collected by the big data collection module, then extracts features and associates contextual semantics on the pre-processed data, and finally obtains the visual analysis results of the target analysis raw data through a generator. In this way, it is possible to more effectively and deeply mine and utilize the information in the data, capture potential patterns and laws, and finally present them in an intuitive and easy-to-understand way.

[0005] According to one aspect of the present application, there is provided an AI-based big data intelligent analysis platform, which includes:

[0006] The target analysis data collection module is used to obtain the target analysis raw data collected by the big data collection module;

[0007] A target analysis data preprocessing module, used for performing data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data;

[0008] A target analysis data feature encoding module is used to perform feature encoding on the pre-processed target analysis raw data to obtain a target analysis raw data fusion feature vector;

[0009] The visualization analysis result generation module is used to fuse the feature vector based on the target analysis original data and process the visualization analysis result of the target analysis original data.

[0010] According to another aspect of the present application, there is provided a big data intelligent analysis method based on AI, which includes:

[0011] Obtaining target analysis raw data collected by the big data collection module;

[0012] Performing data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data;

[0013] Performing feature encoding on the preprocessed target analysis raw data to obtain a target analysis raw data fusion feature vector;

[0014] Based on the target analysis original data, the feature vector is fused and processed to obtain the visual analysis result of the target analysis original data.

[0015] This application has significant technical effects due to the adoption of the above technical solutions:

[0016] The AI-based big data intelligent analysis platform and method provided in this application preprocesses the target analysis raw data collected by the big data collection module, then extracts features and associates contextual semantics on the preprocessed data, and finally obtains the visual analysis results of the target analysis raw data through a generator. In this way, it is possible to more effectively and deeply mine and utilize the information in the data, capture potential patterns and laws, and finally present them in an intuitive and easy-to-understand way. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 It is a block diagram of an AI-based big data intelligent analysis platform according to an embodiment of the present application.

[0019] Figure 2 It is a block diagram of the target analysis data feature encoding module in the AI-based big data intelligent analysis platform according to an embodiment of the present application.

[0020] Figure 3 It is a block diagram of a target analysis data context-related feature extraction unit in an AI-based big data intelligent analysis platform according to an embodiment of the present application.

[0021] Figure 4 This is a flow chart of an AI-based big data intelligent analysis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0023] In today's digital age, the rapid development of mobile Internet and Internet of Things technologies around the world has led to an explosive growth in the generation and accumulation of data. This trend has not only changed the way enterprises and industries operate, but also challenged traditional data analysis methods. Specifically, traditional big data analysis algorithms are often based on relatively fixed theoretical models and assumptions. These models perform well when processing data with clear structure, moderate dimensions and high quality. However, in the current data environment, data presents unprecedented complexity and diversity. Not only are the dimensions extremely high and the quality is uneven, but they are also often accompanied by a large amount of noise, missing values ​​and other challenging problems. When these characteristics of data exceed the effective processing range of traditional algorithms, their ability to mine potential patterns and reveal deep laws becomes inadequate. As a result, the accuracy and precision of the algorithm are greatly reduced, making it difficult to accurately extract key information with practical application value from massive data. Therefore, an AI-based big data intelligent analysis solution is expected.

[0024] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks has provided new solutions and solutions for big data intelligent analysis.

[0025] Figure 1 FIG. 1 is a block diagram of an AI-based big data intelligent analysis platform according to an embodiment of the present application. Figure 1 As shown, in the AI-based big data intelligent analysis platform 100, it includes: a target analysis data collection module 110, which is used to obtain the target analysis original data collected by the big data collection module; a target analysis data preprocessing module 120, which is used to perform data preprocessing on the target analysis original data to obtain the preprocessed target analysis original data; a target analysis data feature encoding module 130, which is used to perform feature encoding on the preprocessed target analysis original data to obtain a fused feature vector of the target analysis original data; and a visualization analysis result generation module 140, which is used to fuse the feature vector based on the target analysis original data and process to obtain the visualization analysis result of the target analysis original data.

[0026] In an embodiment of the present application, the target analysis data collection module 110 is used to obtain the target analysis raw data collected by the big data acquisition module. It should be understood that the big data acquisition module can obtain data from a variety of sources (such as sensors, social media, user interactions, device logs, etc.). This diversity of data sources enables the analysis to cover a wider range of variables and improve the accuracy of the analysis. It should be noted that the specific source data used by the big data acquisition module should be determined according to the specific usage scenario. In general, the form of data collection by the big data acquisition module includes but is not limited to the use of API interfaces, data crawlers, sensors, etc. The target analysis raw data obtained by the big data acquisition module usually contains multiple types of information, which can vary according to different application scenarios and requirements. But in general, the target analysis raw data may contain the following information: numerical data, time-related data, text data, etc. The collected target analysis raw data is the basis and premise of big data intelligent analysis. Without accurate and complete data, effective analysis and mining cannot be carried out.

[0027] In the embodiment of the present application, the target analysis data preprocessing module 120 is used to perform data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data. It should be understood that the target analysis raw data may come from multiple data sources, and their structures and formats may be the same in each part. Even if the data comes from the same data source, it will contain some incomplete, repeated or abnormal data information. In order to make these data meet the requirements of the subsequent analysis algorithm and improve the analysis efficiency to obtain clear and accurate analysis results, in the technical solution of the present application, it is necessary to perform data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data. In particular, in an embodiment of the present application, a specific implementation method for performing data preprocessing on the target analysis raw data to obtain the preprocessed target analysis raw data is: a. Data cleaning: Use the unique identifier existing in the data to delete duplicate data records in each data source; for missing data, if there are fewer missing values, you can choose to delete the data records, if there are more missing values, you can choose to use interpolation methods to fill in the missing values; for abnormal values, you can choose to delete or replace them with a more reasonable value based on historical records; b. Data conversion: The text, date, value and other forms in the data from different data sources are uniformly converted to facilitate subsequent data analysis; c. Data checking: After each preprocessing step, data verification and checking are required to ensure the integrity and consistency of the data.

[0028] In the embodiment of the present application, the target analysis data feature encoding module 130 is used to perform feature encoding on the pre-processed target analysis raw data to obtain a target analysis raw data fusion feature vector. Specifically, Figure 2FIG. 1 is a block diagram of a target analysis data feature encoding module in an AI-based big data intelligent analysis platform according to an embodiment of the present application. Figure 2 As shown, the target analysis data feature encoding module 130 includes: a target analysis data feature preliminary extraction unit 131, used to perform feature extraction on the preprocessed target analysis original data to obtain a target analysis original data feature vector; a target analysis data context-associated feature extraction unit 132, used to perform context semantic association processing on the target analysis original data feature vector to obtain a target analysis original data context-associated feature vector; a target analysis feature fusion unit 133, used to perform intrinsic fine-grained alignment and fusion based on scale difference elimination on the target analysis original data feature vector and the target analysis original data context-associated feature vector to obtain the target analysis original data fused feature vector.

[0029] In an embodiment of the present application, the target analysis data feature preliminary extraction unit 131 is used to extract features from the pre-processed target analysis raw data to obtain the target analysis raw data feature vector. Specifically, in an embodiment of the present application, the target analysis data feature preliminary extraction unit is used to: pass the pre-processed target analysis raw data through the raw data feature extractor to obtain the target analysis raw data feature vector. It should be understood that after the target analysis raw data is pre-processed, although noise, outliers and other problems are removed, the scale of the data is still very large. Directly analyzing large-scale data not only has high computational complexity, but also causes model overfitting and reduces the accuracy of the analysis. In order to extract features that can sufficiently represent the key information of the data from the pre-processed target analysis data, it is necessary to pass the pre-processed target analysis raw data through the raw data feature extractor in the technical solution of the present application. After being processed by the raw data feature extractor, the key information and its pattern in the data can be well captured, which effectively eliminates the noise and unnecessary complexity in the data, so that the subsequent visualization results will be more accurate and convenient for users to make decisions. The essence of the raw data feature extractor in the present application is a deep separable convolutional neural network model. It is worth mentioning that the deep separable convolutional neural network decomposes the standard convolution operation into two steps: deep convolution and point-by-point convolution. Deep convolution applies the convolution kernel to each input channel independently, while point-by-point convolution uses a 1×1 convolution kernel to combine the output of deep convolution. This decomposition significantly reduces the number of parameters of the model and reduces the computational complexity. And because of the reduction in parameters, the deep separable convolutional neural network requires fewer computing resources and faster computing speed, making it more efficient when processing large-scale data and suitable for running in resource-limited environments.

[0030] In the embodiment of the present application, the target analysis data contextual association feature extraction unit 132 is used to perform contextual semantic association processing on the target analysis raw data feature vector to obtain the target analysis raw data contextual association feature vector. Specifically, Figure 3 FIG. 1 is a block diagram of a target analysis data context-related feature extraction unit in an AI-based big data intelligent analysis platform according to an embodiment of the present application. Figure 3 As shown, the target analysis data context-associated feature extraction unit 132 includes: a vector segmentation subunit 1321, used to perform vector segmentation on the target analysis original data feature vector to obtain multiple target analysis original data feature sub-vectors, wherein the lengths of each target analysis original data feature sub-vector in the multiple target analysis original data feature sub-vectors are equal; a context-associated feature generation subunit 1322, used to pass the multiple target analysis original data feature sub-vectors through a context-associated feature encoder to obtain the target analysis original data context-associated feature vector.

[0031] In an embodiment of the present application, the vector segmentation subunit 1321 is used to perform vector segmentation on the target analysis original data feature vector to obtain multiple target analysis original data feature subvectors, wherein the lengths of the target analysis original data feature subvectors in the multiple target analysis original data feature subvectors are equal. It should be understood that the target analysis original data feature vector usually contains different information in multiple aspects. In order to help analyze the different aspects of the data and the correlation between the different aspects in more detail, the target analysis original data feature vector can be vector segmented. The lengths of the segmented subvectors are equal, which can ensure that in subsequent processing and analysis, all subvectors have the same feature dimensions and structures. This uniformity avoids the subsequent computational complexity caused by inconsistent dimensions. Moreover, the segmented feature subvectors can be processed in parallel, which makes full use of the multi-core parallel computing capabilities of modern computing resources and is conducive to improving data processing efficiency.

[0032] In an embodiment of the present application, the context-related feature generation subunit 1322 is used to pass the multiple target analysis raw data feature subvectors through the context-related feature encoder to obtain the target analysis raw data context-related feature vectors. It should be understood that the segmented feature subvectors may independently represent different information, but in practical applications, these features are often interrelated. In order to learn the relationship between features and capture richer context information, thereby improving the performance of the model, the multiple target analysis raw data feature subvectors can be input into the context-related feature encoder in the technical solution of the present application. The essence of the context-related feature encoder in the present application is a context-related feature encoder based on a converter. The converter (Transformer) is a deep learning model architecture that can dynamically focus on different parts of the sequence when processing sequence input data through a self-attention mechanism, which enables the model to capture long-distance dependencies, thereby better understanding context information, and unlike traditional recursive neural networks, the converter can process input data in parallel, improving the efficiency of training and reasoning. The target analysis raw data context-related feature vector obtained by processing can provide more comprehensive feature information, making the subsequent generated visual analysis results richer and more detailed.

[0033] Specifically, in an embodiment of the present application, the multiple target analysis raw data feature sub-vectors are passed through a context-associated feature encoder to obtain the target analysis raw data context-associated feature vector, including: arranging the multiple target analysis raw data feature sub-vectors in one dimension to obtain a target analysis raw data global feature vector; calculating the product between the global feature vector of the target analysis raw data and the transposed vector of each target analysis raw data feature sub-vector in the multiple target analysis raw data feature sub-vectors to obtain multiple target analysis raw data self-attention association matrices; and respectively performing one-dimensional arranging on the multiple target analysis raw data self-attention association matrices of each target analysis raw data self-attention association matrix. The association matrix is ​​standardized to obtain a plurality of standardized target analysis raw data self-attention association matrices; each standardized target analysis raw data self-attention association matrix in the plurality of standardized target analysis raw data self-attention association matrices is subjected to a Softmax classification function to obtain a plurality of probability values; each target analysis raw data feature subvector in the plurality of target analysis raw data feature subvectors is weighted using each probability value in the plurality of probability values ​​as a weight to obtain a plurality of target analysis raw data weighted feature subvectors; the plurality of target analysis raw data weighted feature subvectors are cascaded to obtain the target analysis raw data context association feature vector.

[0034] In an embodiment of the present application, the target analysis feature fusion unit 133 is used to perform intrinsic fine-grained alignment fusion based on scale difference elimination on the target analysis raw data feature vector and the target analysis raw data context-associated feature vector to obtain the target analysis raw data fused feature vector. It should be understood that the target analysis raw data feature vector usually contains key feature information extracted from the raw data, while the target analysis raw data context-associated feature vector contains the correlation and context information between the data. In order to comprehensively utilize the key features and context information in the raw data to improve the accuracy and reliability of the analysis, it is necessary in the technical solution of the present application to perform intrinsic fine-grained alignment fusion based on scale difference elimination on the target analysis raw data feature vector and the target analysis raw data context-associated feature vector.

[0035] In particular, considering that the target analysis raw data feature vector usually contains the original features extracted directly from the data, while the target analysis raw data context-associated feature vector contains more semantic information and contextual relationships, this information is often obtained through advanced data processing and feature extraction techniques. Due to different processing methods, the two vectors have specific different scale ranges. Such different scale ranges will cause the importance of features in subsequent processing to be misestimated. And when directly fusing the target analysis raw data feature vector and the target analysis raw data context-associated feature vector, if they contain the same information or are highly correlated, it will lead to information conflict and overlap, and then some important information will be ignored, that is, the problem of information loss will occur, which will affect the accuracy of subsequent visual analysis results. Therefore, in the technical solution of the present application, it is necessary to perform intrinsic fine-grained alignment fusion of the target analysis raw data feature vector and the target analysis raw data context-associated feature vector based on scale difference elimination to obtain the target analysis raw data fused feature vector.

[0036] Specifically, in an embodiment of the present application, the target analysis raw data feature vector and the target analysis raw data context-associated feature vector are subjected to intrinsic fine-grained alignment and fusion based on scale difference elimination to obtain the target analysis raw data fused feature vector, including: performing normalization modulation based on vector norming on the target analysis raw data feature vector and the target analysis raw data context-associated feature vector to obtain a pre-aligned target analysis raw data feature vector and a pre-aligned target analysis raw data context-associated feature vector; performing intrinsic feature extraction on the pre-aligned target analysis raw data feature vector and the pre-aligned target analysis raw data context-associated feature vector to obtain a first target analysis raw data principal component feature vector and a second target analysis raw data principal component feature vector; performing intrinsic fine-grained alignment and fusion on the first target analysis raw data principal component feature vector and the second target analysis raw data principal component feature vector to obtain the target analysis raw data fused feature vector.

[0037] In the embodiment of the present application, specifically, another implementation expression of performing intrinsic fine-grained alignment fusion based on scale difference elimination on the target analysis original data feature vector and the target analysis original data context-related feature vector to obtain the target analysis original data fusion feature vector can be: processing the target analysis original data feature vector and the target analysis original data context-related feature vector according to the following formula to obtain the target analysis original data fusion feature vector; wherein the formula is:

[0038]

[0039] Among them, V1 represents the target analysis original data feature vector, V2 represents the target analysis original data context association feature vector, v 1i represents the eigenvalue of the i-th position of the target analysis raw data eigenvector, v 2i represents the eigenvalue of the i-th position of the context-related eigenvector of the target analysis raw data, exp represents the natural exponential function, represents the square of the bi-norm of the vector, V1' represents the original data feature vector of the pre-aligned target analysis, V2' represents the context-related feature vector of the original data of the pre-aligned target analysis, PCA(·) represents the intrinsic feature extraction, U1 represents the sequence of the first intrinsic decomposition vector, Λ1 represents the first diagonal matrix, λ 11 and λ 1m denote the eigenvalues ​​of the first and mth positions of the first diagonal matrix, respectively. T denotes the transpose of the vector. concat{·;·} denotes vector concatenation. v 11 、v 12 、v 1mRespectively represent the first, second and mth eigenvectors of the sequence of the principal component eigenvectors of the original data of the first target analysis, V k represents the principal component eigenvector of the original data of the first target analysis, U2 represents the sequence of the second eigendecomposition vector, Λ2 represents the second diagonal matrix, λ 21 and λ 2m denote the eigenvalues ​​of the first and mth positions of the second diagonal matrix, respectively, v 21 、v 22 、v 2m Respectively represent the first, second and mth eigenvectors of the sequence of the principal component eigenvectors of the second target analysis original data, V s represents the principal component eigenvector of the original data for the second objective analysis, represents positional subtraction, ⊙ represents positional multiplication, Indicates positional addition, Conv 1×1 represents point convolution, f vp (V k ,V s ) represents the target analysis raw data fusion feature vector.

[0040] That is, in the technical solution of the present application, the target analysis original data feature vector and the target analysis original data context-related feature vector are subjected to intrinsic fine-grained alignment fusion based on scale difference elimination to obtain the target analysis original data fusion feature vector, which first performs normalization modulation based on vector norming on the target analysis original data feature vector and the target analysis original data context-related feature vector to obtain the pre-aligned target analysis original data feature vector and the pre-aligned target analysis original data context-related feature vector. This process is not just for simply adjusting the scale, but an important technology in feature engineering, which aims to solve the problem of inconsistent scales between features in the data set. By calculating the modulus of the feature vector and dividing each vector by its own modulus, it can be ensured that all feature vectors have the same length, thereby eliminating the impact of scale differences between different feature vectors. Normalization processing not only contributes to numerical stability in subsequent processing, but also makes the comparison between feature vectors more fair and reasonable, laying the foundation for subsequent feature alignment. In machine learning models, especially algorithms such as neural networks, the scale of the input features directly affects the speed and direction of weight updates, so normalization can be regarded as a regularization method, which helps to prevent overfitting and also helps to avoid the problem of gradient disappearance or explosion.

[0041] Next, the pre-aligned target analysis raw data feature vector and the pre-aligned target analysis raw data context-associated feature vector are subjected to intrinsic feature extraction to obtain the first target analysis raw data principal component feature vector and the second target analysis raw data principal component feature vector. That is, after the normalization modulation is completed, the pre-aligned target analysis raw data feature vector and the target analysis raw data context-associated feature vector are subjected to intrinsic feature extraction. As an unsupervised linear dimensionality reduction method, intrinsic feature extraction is widely used in data compression and feature extraction. This process determines the principal components by finding the direction of the maximum variance of the data. These principal components are new coordinate axes in the original feature space, which can capture the structure of the data with minimal information loss. It is worth mentioning that by selecting the eigenvectors corresponding to the first few maximum eigenvalues, the dimension of the feature space can be effectively reduced while maintaining the key information of the data. This dimensionality reduction can not only accelerate the model training process, but also help the model to generalize better, reduce the impact of noise, and thus improve the performance of the model.

[0042] Finally, the principal component feature vector of the first target analysis raw data and the principal component feature vector of the second target analysis raw data are intrinsically aligned and fused to obtain the target analysis raw data fusion feature vector. The goal of this step is to find the best matching point or area between the two target analysis raw data principal component feature vectors to achieve alignment between the two. Fine-grained alignment goes beyond the traditional rough fusion methods such as simple weighted average or splicing, and needs to consider the time series characteristics or spatial distribution characteristics of the feature vector. The fusion after alignment is not a simple numerical operation, but requires the design of a specific mechanism to determine how to combine the information of the two feature vectors. This content-based intelligent fusion method can more effectively utilize multi-source information and improve the representation ability and decision-making quality of the model.

[0043] In an embodiment of the present application, the visualization analysis result generation module 140 is used to fuse the feature vector based on the target analysis raw data, and process to obtain the visualization analysis result of the target analysis raw data. Specifically, in an embodiment of the present application, the visualization analysis result generation module is used to: fuse the target analysis raw data with the feature vector through a generator to obtain a generation result, and the generation result is a visualization analysis result of the target analysis raw data. It should be understood that the generator here refers to an algorithm model that can convert a feature vector into a visualization result. It usually receives a processed feature vector as input and outputs one or more visualization elements (such as images, charts, animations, etc.), which intuitively show the inherent structure and laws of the target analysis raw data. The generator can be constructed based on advanced technologies such as deep learning, generative adversarial networks (GANs), and variational autoencoders (VAEs), and this application does not specifically limit this. The generator can transform complex feature vectors into intuitive and easy-to-understand visualization results, which helps users understand the data more intuitively and discover potential rules and patterns. The generator can automatically generate visualization results, reducing the time and cost of manual analysis and interpretation of data. The visualization analysis results can also provide intuitive data support for decision makers, helping them make more informed and scientific decisions.

[0044] In summary, the AI-based big data intelligent analysis platform 100 based on the embodiment of the present application is explained, which pre-processes the target analysis raw data collected by the big data collection module, then extracts features and associates contextual semantics on the pre-processed data, and finally obtains the visual analysis results of the target analysis raw data through the generator. In this way, it is possible to more effectively and deeply mine and utilize the information in the data, capture potential patterns and laws, and finally present them in an intuitive and easy-to-understand manner.

[0045] As described above, the AI-based big data intelligent analysis platform 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for AI-based big data intelligent analysis. In one example, the AI-based big data intelligent analysis platform 100 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the AI-based big data intelligent analysis platform 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the AI-based big data intelligent analysis platform 100 can also be one of the many hardware modules of the terminal device.

[0046] Alternatively, in another example, the AI-based big data intelligent analysis platform 100 and the terminal device may also be separate devices, and the AI-based big data intelligent analysis platform 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0047] Figure 4 Flowchart of the AI-based big data intelligent analysis method according to an embodiment of the present application. Figure 4 As shown, in the AI-based big data intelligent analysis method, it includes: S110, acquiring the target analysis original data collected by the big data acquisition module; S120, performing data preprocessing on the target analysis original data to obtain preprocessed target analysis original data; S130, performing feature encoding on the preprocessed target analysis original data to obtain a fused feature vector of the target analysis original data; S140, based on the fused feature vector of the target analysis original data, processing to obtain a visual analysis result of the target analysis original data.

[0048] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned AI-based big data intelligent analysis method have been referred to above. Figures 1 to 3 The description of the AI-based big data intelligent analysis platform has been introduced in detail, and therefore, its repeated description will be omitted.

[0049] In summary, the AI-based big data intelligent analysis method based on the embodiment of the present application is explained, which pre-processes the target analysis raw data collected by the big data acquisition module, then extracts features and associates contextual semantics on the pre-processed data, and finally obtains the visual analysis results of the target analysis raw data through the generator. In this way, it is possible to more effectively and deeply explore and utilize the information in the data, capture potential patterns and laws, and finally present them in an intuitive and easy-to-understand way.

[0050] Finally, the present application also relates to a computer-readable storage medium for implementing an AI-based big data intelligent analysis method according to one or more embodiments of the present application. The computer-readable storage medium mentioned herein includes various types of computer storage media, which may be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, a computer-readable storage medium may include RAM, ROM, EPROM, E2PROM, registers, hard disks, removable disks, CD-ROMs or other optical disk storage, disk storage or other magnetic storage devices, or any other temporary or non-temporary medium that can be used to carry or store desired program code units in the form of instructions or data structures and can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. The description of the computer-readable storage medium according to the present application can refer to the explanation of the AI-based big data intelligent analysis method according to the present application, which will not be repeated here.

Claims

1. An AI-based big data intelligent analysis platform, characterized in that: include: The target analysis data collection module is used to obtain the target analysis raw data collected by the big data collection module; A target analysis data preprocessing module, used for performing data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data; A target analysis data feature encoding module is used to perform feature encoding on the pre-processed target analysis raw data to obtain a target analysis raw data fusion feature vector; The visualization analysis result generation module is used to fuse the feature vector based on the target analysis original data and process the visualization analysis result of the target analysis original data.

2. The AI-based big data intelligent analysis platform according to claim 1, characterized in that: The target analysis data feature encoding module includes: A target analysis data feature preliminary extraction unit, used for extracting features from the preprocessed target analysis raw data to obtain a target analysis raw data feature vector; A target analysis data context association feature extraction unit, used for performing context semantic association processing on the target analysis original data feature vector to obtain a target analysis original data context association feature vector; The target analysis feature fusion unit is used to perform intrinsic fine-grained alignment fusion on the target analysis original data feature vector and the target analysis original data context-related feature vector based on scale difference elimination to obtain the target analysis original data fused feature vector.

3. The AI-based big data intelligent analysis platform according to claim 2, characterized in that: The target analysis data feature preliminary extraction unit is used to: pass the pre-processed target analysis raw data through a raw data feature extractor to obtain the target analysis raw data feature vector.

4. The AI-based big data intelligent analysis platform according to claim 3 is characterized in that: The target analysis data context association feature extraction unit comprises: A vector segmentation subunit, used for performing vector segmentation on the target analysis original data feature vector to obtain a plurality of target analysis original data feature sub-vectors, wherein the lengths of the target analysis original data feature sub-vectors in the plurality of target analysis original data feature sub-vectors are equal; The context-associated feature generating subunit is used to pass the multiple target analysis original data feature sub-vectors through a context-associated feature encoder to obtain the target analysis original data context-associated feature vector.

5. The AI-based big data intelligent analysis platform according to claim 4 is characterized in that: The raw data feature extractor is a deep separable convolutional neural network model, and the context-related feature encoder is a converter-based context-related feature encoder.

6. The AI-based big data intelligent analysis platform according to claim 5, characterized in that: The target analysis feature fusion unit is used to: Performing normalization modulation based on vector norming on the target analysis original data feature vector and the target analysis original data context association feature vector to obtain a pre-aligned target analysis original data feature vector and a pre-aligned target analysis original data context association feature vector; Performing intrinsic feature extraction on the pre-aligned target analysis original data feature vector and the pre-aligned target analysis original data context association feature vector to obtain a first target analysis original data principal component feature vector and a second target analysis original data principal component feature vector; The principal component feature vector of the first target analysis original data and the principal component feature vector of the second target analysis original data are intrinsically aligned and fused to obtain the target analysis original data fused feature vector.

7. The AI-based big data intelligent analysis platform according to claim 6, characterized in that: The visualization analysis result generation module is used to: fuse the target analysis original data with the feature vector through a generator to obtain a generation result, and the generation result is a visualization analysis result of the target analysis original data.

8. A big data intelligent analysis method based on AI, characterized in that: include: Obtaining target analysis raw data collected by the big data collection module; Performing data preprocessing on the target analysis raw data to obtain preprocessed target analysis raw data; Performing feature encoding on the preprocessed target analysis raw data to obtain a target analysis raw data fusion feature vector; Based on the target analysis original data, the feature vector is fused and processed to obtain the visual analysis result of the target analysis original data.

9. The AI-based big data intelligent analysis method according to claim 8, characterized in that: Performing feature encoding on the preprocessed target analysis raw data to obtain a target analysis raw data fusion feature vector, including: Performing feature extraction on the preprocessed target analysis raw data to obtain a target analysis raw data feature vector; Performing context semantic association processing on the target analysis original data feature vector to obtain a target analysis original data context association feature vector; The target analysis original data feature vector and the target analysis original data context-related feature vector are subjected to intrinsic fine-grained alignment fusion based on scale difference elimination to obtain the target analysis original data fused feature vector.

10. The AI-based big data intelligent analysis method according to claim 9, characterized in that: Based on the target analysis original data fused with feature vectors, processing is performed to obtain visualization analysis results of the target analysis original data, including: passing the target analysis original data fused with feature vectors through a generator to obtain a generation result, wherein the generation result is the visualization analysis result of the target analysis original data.

Images