Online data processing method for business intelligent service platform

Abstract

The invention discloses an online data processing method for a business intelligent service platform, and relates to the technical field of big data processing and artificial intelligence, and the method comprises the steps: collecting an original data flow, and generating a standardized data set; extracting spatio-temporal features from the standardized data set by using a deep convolutional neural network model to form a feature combination matrix; constructing a dynamic association graph for the feature combination matrix based on a graph attention network, and calculating association strength among graph nodes by adopting an edge probability calculation model to generate an association weight matrix; a self-adaptive learning rate mechanism is introduced, weight parameters are dynamically updated in combination with an exponential moving average method, and a feature propagation network is established; and inputting the incremental business data into the feature propagation network, predicting a data change trend through an ensemble learning algorithm, outputting a data analysis result, and feeding back the data analysis result to the business intelligent service platform. According to the method, a brand new breakthrough of business data processing is realized through fusion of advanced technologies of deep learning feature extraction and graph network association analysis.

Description

Technical Field

[0001] The present invention relates to the field of big data processing and artificial intelligence technology, and in particular to an online data processing method for a commercial intelligence service platform. Background Art

[0002] Traditional business intelligence service platforms mainly rely on static data processing and simple statistical analysis methods, which are difficult to cope with the rapidly changing business environment and the real-time processing challenges of massive multi-dimensional data. These platforms usually face technical bottlenecks such as low data collection efficiency, inaccurate feature extraction, and limited data correlation analysis capabilities, which restrict enterprises from deeply mining data value and making strategic decisions scientifically.

[0003] The technical architecture of existing business intelligence service platforms generally has key problems such as non-real-time data processing, insufficient granularity of feature extraction, and lack of dynamic data correlation analysis. Traditional methods often use a single data processing model, which cannot effectively capture the spatiotemporal complexity and potential correlation characteristics of business data. Especially when faced with cross-domain, multi-source heterogeneous data, existing technologies find it difficult to build flexible and intelligent data correlation networks, and lack adaptive learning mechanisms, which greatly reduces the accuracy and timeliness of data analysis. These technical limitations seriously hinder the full release of the value of business intelligence service platforms for enterprise decision-making, and there is an urgent need to break through the traditional technical paradigm and build a more intelligent and efficient data processing and analysis paradigm.

[0004] In view of the many limitations of existing business intelligence service platforms in data processing technology, the present invention proposes an online data processing method for business intelligence service platforms, aiming to solve the key technical problems of low data processing efficiency and inaccurate correlation analysis of traditional business intelligence service platforms; by innovatively integrating distributed data collection, deep learning feature extraction, graph network correlation analysis and other cutting-edge technologies, a high-efficiency business data intelligent processing technology solution is constructed. Summary of the invention

[0005] In view of the problems existing in the data processing technology of the existing commercial intelligence service platform, the present invention is proposed.

[0006] Therefore, the problem to be solved by the present invention is how to achieve accurate feature extraction, dynamic correlation analysis and real-time trend prediction of multi-source heterogeneous data, and break through the technical bottlenecks of traditional business intelligence service platforms in data processing efficiency, feature correlation and adaptive learning capabilities.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In the first aspect, an embodiment of the present invention provides an online data processing method for a business intelligence service platform, which includes collecting the original data stream of the business intelligence service platform through a distributed collection gateway, preprocessing the original data, and generating a normalized data set; using a deep convolutional neural network model to extract spatiotemporal features from the normalized data set, and combining wavelet transform to decompose the multi-scale components of the spatiotemporal features to form a feature combination matrix; constructing a dynamic association graph for the feature combination matrix based on a graph attention network, and using an edge probability calculation model to calculate the association strength between graph nodes to generate an association weight matrix; introducing an adaptive learning rate mechanism to the association weight matrix, and dynamically updating the weight parameters in combination with the exponential sliding average method to establish a feature propagation network; inputting incremental business data into the feature propagation network, predicting data change trends through an integrated learning algorithm, outputting data analysis results, and feeding back to the business intelligence service platform.

[0009] As a preferred solution of the online data processing method for the business intelligence service platform described in the present invention, wherein: the incremental business data is input into the feature propagation network, the data change trend is predicted by the integrated learning algorithm, the data analysis results are output, and the results are fed back to the business intelligence service platform, including: preprocessing the incremental business data, and mapping the preprocessed incremental business data to the input layer nodes of the feature propagation network, extracting and fusing feature information, and generating high-dimensional feature data; constructing several sub-models according to the high-dimensional feature data, wherein the sub-models include a gradient boosting tree model and a random forest model; and calculating the comprehensive prediction score based on the output results of the sub-models. The specific formula is as follows:

[0010]

[0011] Among them, S final is the comprehensive prediction score, is the gradient boosting tree model weight, is the random forest model weight, M1 is the number of gradient boosting trees, M2 is the number of random forests, G i is the prediction result of the i-th gradient boosting tree, y i is the confidence of the i-th gradient boosting tree, R j is the prediction result of the jth random forest, z j is the confidence of the jth random forest, and δ is the bias correction term of the sub-model.

[0012] The data analysis results are output according to the comprehensive prediction score and transmitted to the data display module of the business intelligence service platform to adjust the business parameters and strategy configuration, wherein the data analysis results include data change trend charts, abnormal fluctuation detection results and key indicator warning information.

[0013] As a preferred solution of the online data processing method for the business intelligence service platform described in the present invention, wherein: the comprehensive prediction score adopts a weighted voting method to fuse the prediction results of each sub-model; the method for establishing the feature propagation network is to set an initial learning rate parameter for the associated weight matrix, and calculate the adaptive adjustment factor based on the weight gradient change trend, wherein the adaptive adjustment factor is dynamically adjusted as the weight gradient changes; in each round of iteration, the exponential sliding average method is used to calculate the historical cumulative weight of the associated weight matrix; the current weight parameter is updated based on the product of the adaptive adjustment factor and the historical cumulative weight to form an optimized weight coefficient; the weight coefficient is input into the feature propagation layer to construct a feature propagation network with dynamic weight optimization capability.

[0014] As a preferred solution of the online data processing method for a business intelligence service platform described in the present invention, wherein: the method for generating the association weight matrix is ​​to construct an initial node set in the graph attention network according to the feature combination matrix, and input the initial node set into the multi-head attention layer to calculate the attention coefficient between nodes, wherein the nodes in the initial node set represent feature vectors; construct a dynamic connection relationship between nodes according to the attention coefficient, and use a nonlinear activation function for the dynamic connection relationship to generate a dynamic association graph; use an edge probability calculation model to obtain the transition probability between node pairs in the dynamic association graph, wherein the transition probability represents the degree of association between nodes; perform weighted calculation on the node pairs according to the transition probability to generate an association weight matrix.

[0015] As a preferred solution of the online data processing method for the business intelligence service platform described in the present invention, wherein: the method for forming the feature combination matrix is ​​to segment the normalized data set according to the time series, construct a sliding time window, and extract the spatiotemporal features of the data in each time window through a deep convolutional neural network model; select a wavelet basis function to perform discrete wavelet transform on the spatiotemporal features, decompose the characteristic signal into sub-components of different frequency bands, calculate the energy distribution characteristics of the sub-components of each frequency band, and extract characteristic coefficients; based on the characteristic coefficients, construct a multi-scale feature descriptor, and fuse the multi-scale feature descriptor and the transformed spatiotemporal features; use a feature mapping algorithm to project features from different sources into the feature space to generate a feature combination matrix.

[0016] As a preferred solution of the online data processing method for the business intelligence service platform described in the present invention, the specific formula of the feature combination matrix is ​​as follows:

[0017]

[0018] Among them, M combined is the feature combination matrix, B is the feature dimension, γi is the feature weight of the i-th dimension, F i is the eigenvalue of the i-th dimension, μ i is the feature mean of the i-th dimension, τ i is the standard deviation of the i-th dimension feature, λ is the smoothing factor, K is the number of sampling windows, ω k is the kth window weight, H k is the entropy value of the kth window, tanh(Δ) is the feature difference, E dist is the energy distribution characteristic value, and the specific formula is as follows:

[0019]

[0020] Among them, E dist is the energy distribution eigenvalue, J is the number of wavelet decomposition layers, β j is the energy weight coefficient of the jth layer, W j is the j-th layer wavelet coefficient, N is the number of frequency sampling points, P(f) is the power spectrum density at frequency f, ΔE is the energy difference between adjacent frequency bands, and S is the spectrum entropy.

[0021] As a preferred solution of the online data processing method for the business intelligence service platform described in the present invention, wherein: the deep convolutional neural network model includes a convolution layer, a pooling layer and a fully connected layer; the convolution layer uses convolution kernels of different scales to extract local features; the pooling layer reduces the feature dimension through the maximum pooling operation; the fully connected layer integrates the feature outputs of each layer; the method for forming the normalized data set is to collect the original data stream from each business system interface of the business intelligence service platform through a distributed acquisition gateway; based on the preset data verification rules, the format verification and integrity check of the original data stream are performed to eliminate data records that do not meet the specifications; the data cleaning algorithm is used to process the abnormal values, duplicate values ​​and missing values ​​in the original data stream; the cleaned data is subjected to field mapping and format conversion according to the data structure standard to generate standardized data records; the data records are classified and sorted according to the business type to form a normalized data set, and efficient data access services are provided through the data indexing mechanism, and a data version control mechanism is set to record the data processing history.

[0022] In the second aspect, an embodiment of the present invention provides an online data processing system for a business intelligence service platform, which includes: a data acquisition module, which is used to collect the original data stream of the business intelligence service platform through a distributed acquisition gateway, pre-process the original data, and generate a normalized data set; a feature extraction module, which is used to extract spatiotemporal features from the normalized data set using a deep convolutional neural network model, and decompose the multi-scale components of the spatiotemporal features in combination with wavelet transform to form a feature combination matrix; a dynamic association graph construction module, which constructs a dynamic association graph for the feature combination matrix based on a graph attention network, and uses an edge probability calculation model to calculate the association strength between graph nodes to generate an association weight matrix; an establishment module, which is used to introduce an adaptive learning rate mechanism to the association weight matrix, dynamically update the weight parameters in combination with the exponential sliding average method, and establish a feature propagation network; a feedback module, which is used to input incremental business data into the feature propagation network, predict data change trends through an integrated learning algorithm, output data analysis results, and feed back to the business intelligence service platform.

[0023] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the online data processing method for a business intelligence service platform as described in the first aspect of the present invention are implemented.

[0024] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the online data processing method for a business intelligence service platform as described in the first aspect of the present invention are implemented.

[0025] The beneficial effects of the present invention are as follows: the present invention accurately collects multi-source heterogeneous data through a distributed collection gateway, and generates a high-quality normalized data set after strict preprocessing; utilizes deep convolutional neural networks and wavelet transforms to extract data spatiotemporal features and enhance the depth and breadth of feature expression; constructs a dynamic association graph through a graph attention network to characterize the complex association relationship between data nodes, and introduces an adaptive learning rate mechanism to achieve real-time dynamic optimization of the feature propagation network; through an integrated learning algorithm, it can not only accurately predict data change trends, but also output multi-dimensional analysis results including change trend graphs, abnormal fluctuation detection, and key indicator warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0027] Figure 1 This is a flow chart of an online data processing method for a business intelligence service platform according to Example 1. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0030] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0031] Example 1

[0032] Reference Figure 1 , which is the first embodiment of the present invention, and provides an online data processing method for a business intelligence service platform, comprising:

[0033] S1: Collect the original data stream of the business intelligence service platform through the distributed collection gateway, pre-process the original data, and generate a normalized data set.

[0034] Specifically, the method for forming the normalized data set is to collect the original data stream from each business system interface of the business intelligence service platform through a distributed collection gateway.

[0035] It should be noted that the business system includes order management system, customer relationship management system, inventory management system and financial accounting system. The distributed collection gateway includes data access module, task scheduling module and data cache module; the data access module is responsible for establishing a secure connection channel with each business system; the task scheduling module allocates computing resources according to the preset collection strategy; the data cache module temporarily stores the collected data.

[0036] Furthermore, based on preset data verification rules, the original data stream is format checked and checked for integrity, and data records that do not meet the specifications are eliminated; a data cleaning algorithm is used to process outliers, duplicate values, and missing values ​​in the original data stream; and field mapping and format conversion are performed on the cleaned data in accordance with data structure standards to generate standardized data records.

[0037] Furthermore, data records are classified and organized according to business types to form standardized data sets, and efficient data access services are provided through a data indexing mechanism. At the same time, a data version control mechanism is set up to record the data processing history.

[0038] S2: Using a deep convolutional neural network model to extract spatiotemporal features from the normalized data set, and combining wavelet transform to decompose the multi-scale components of the spatiotemporal features to form a feature combination matrix.

[0039] Specifically, the method for forming the feature combination matrix is ​​to segment the normalized data set according to the time series, construct a sliding time window, and extract the spatiotemporal features of the data in each time window through a deep convolutional neural network model; select the wavelet basis function to perform discrete wavelet transform on the spatiotemporal features, decompose the characteristic signal into sub-components in different frequency bands, calculate the energy distribution characteristics of the sub-components in each frequency band, and extract the characteristic coefficients.

[0040] Furthermore, the relevant formula for energy distribution characteristics is as follows:

[0041]

[0042] Among them, E dist is the energy distribution eigenvalue, J is the number of wavelet decomposition layers, β j is the energy weight coefficient of the jth layer, W j is the j-th layer wavelet coefficient, N is the number of frequency sampling points, P(f) is the power spectrum density at frequency f, ΔE is the energy difference between adjacent frequency bands, and S is the spectrum entropy.

[0043] It should be noted that if E dist >0.8, the characteristic signal energy is concentrated and high-frequency feature extraction is started; if 0.4≤E dist ≤0.8, the energy distribution is balanced and the current sampling frequency is maintained; if E dist <0.4, the signal energy is dispersed and the sampling density is increased.

[0044] Furthermore, based on the feature coefficients, a multi-scale feature descriptor is constructed, and the multi-scale feature descriptor and the transformed spatiotemporal features are fused; the feature mapping algorithm is used to project the features from different sources into the feature space to generate a feature combination matrix. The specific formula is as follows:

[0045]

[0046] Among them, M combined is the feature combination matrix, B is the feature dimension, γ i is the feature weight of the i-th dimension, F i is the eigenvalue of the i-th dimension, μ i is the feature mean of the i-th dimension, τ i is the standard deviation of the i-th dimension feature, λ is the smoothing factor, K is the number of sampling windows, ω k is the kth window weight, H k is the entropy value of the kth window, tanh(Δ) is the feature difference, E dist is the energy distribution characteristic value.

[0047] It should be noted that if M combined >0.4, the features are strongly correlated and the current feature combination is retained; if -0.4≤M combined When ≤0.4, the features are weakly correlated and the feature weights are adjusted. combined When <-0.4, the features are negatively correlated and the feature map is recalculated.

[0048] S3: Based on the graph attention network, a dynamic association graph is constructed for the feature combination matrix, and the edge probability calculation model is used to calculate the association strength between the graph nodes to generate an association weight matrix.

[0049] Specifically, the method for generating the association weight matrix is ​​to construct an initial node set in the graph attention network according to the feature combination matrix, and input the initial node set into the multi-head attention layer to calculate the attention coefficient between nodes, where the nodes in the initial node set represent feature vectors.

[0050] Furthermore, a dynamic connection relationship of nodes is constructed according to the attention coefficient, and a nonlinear activation function is applied to the dynamic connection relationship to generate a dynamic association graph; an edge probability calculation model is used to obtain the transition probability between node pairs in the dynamic association graph, wherein the transition probability represents the degree of association between nodes;

[0051] It should be noted that the edge probability calculation model integrates the concept of node correlation in graph theory, information propagation theory and the mathematical principle of probability transfer matrix. By introducing nonlinear activation functions and probability transfer calculations, it can accurately characterize the correlation strength between nodes in complex networks.

[0052] Furthermore, the node pairs are weighted according to the transition probability to generate an association weight matrix. The specific formula is as follows:

[0053]

[0054] Among them, W rel is the final association weight matrix, H is the number of attention heads, α h is the weight coefficient of the hth attention head, σ is the nonlinear activation function, V i and V j are the feature vectors of node i and node j respectively, W h is the parameter matrix of the h-th attention head, B is the feature dimension, D is the number of transfer steps, is the transition probability from node i to node j, and n is the total number of nodes.

[0055] It should be noted that the association weight matrix formula uses a multi-head attention layer to capture the association patterns of different feature dimensions, and then combines a nonlinear activation function to process the dynamic connection relationship between nodes, and finally obtains the final weight distribution through the normalized calculation of the transition probability.

[0056] S4: Introduce an adaptive learning rate mechanism to the association weight matrix, dynamically update the weight parameters in combination with the exponential moving average method, and establish a feature propagation network.

[0057] Specifically, the method for establishing the feature propagation network is to set the initial learning rate parameter for the associated weight matrix, calculate the adaptive adjustment factor based on the weight gradient change trend, where the adaptive adjustment factor is dynamically adjusted as the weight gradient changes; in each round of iteration, the exponential sliding average method is used to calculate the historical cumulative weight of the associated weight matrix;

[0058] Furthermore, the current weight parameter is updated based on the product of the adaptive adjustment factor and the historical accumulated weight to form an optimized weight coefficient; the weight coefficient is input into the feature propagation layer to construct a feature propagation network with dynamic weight optimization capability.

[0059] S5: Input the incremental business data into the feature propagation network, predict the data change trend through the integrated learning algorithm, output the data analysis results, and feed them back to the business intelligence service platform.

[0060] Specifically, the incremental business data is preprocessed and mapped to the input layer nodes of the feature propagation network to extract and fuse feature information to generate high-dimensional feature data; and several sub-models are constructed based on the high-dimensional feature data.

[0061] It should be noted that the sub-models include the gradient boosting tree model and the random forest model; the gradient boosting tree model is an ensemble learning algorithm based on the gradient boosting method. Its core idea is to construct a series of weak classifiers or regression trees in an iterative manner. Each iteration attempts to reduce the residual of the previous round of models, calculate the negative gradient of the prediction results of the previous round of models, and use it as the training target of the new tree, so as to gradually optimize the prediction performance of the model. The random forest model is an ensemble classification algorithm based on ensemble learning and decision tree theory. Its basic principle is to construct multiple decision trees, randomly select feature subsets and sample subsets during the training process of each tree, and then vote or average the prediction results of these decision trees to obtain the final prediction results.

[0062] Furthermore, the comprehensive prediction score is calculated based on the output results of the sub-models. The specific formula is as follows:

[0063]

[0064] Among them, S final is the comprehensive prediction score, is the gradient boosting tree model weight, is the random forest model weight, M1 is the number of gradient boosting trees, M2 is the number of random forests, G i is the prediction result of the i-th gradient boosting tree, y i is the confidence of the i-th gradient boosting tree, R j is the prediction result of the jth random forest, z j is the confidence of the jth random forest, and δ is the bias correction term of the sub-model.

[0065] Furthermore, when S final When S ≥ 0.7, the high-risk warning mechanism is triggered, and the warning information is immediately pushed to the business intelligence platform, and it is recommended to adjust to a conservative strategy; when 0.4 ≤ S final <0.7, medium risk monitoring is started, the platform status is updated every hour, and the existing parameter configuration is maintained; when S final <0.4, routine monitoring is performed, data is summarized daily, and parameters can be optimized appropriately; when |G i -R j |>0.3, the model consistency check is started, and the score is recalculated by increasing the δ value for correction; when S final When the change exceeds 0.2, the trend analysis module is started and a report is generated.

[0066] It should be noted that the comprehensive prediction score adopts a weighted voting method to integrate the prediction results of each sub-model; the data analysis results are output according to the comprehensive prediction score and transmitted to the data display module of the business intelligence service platform to adjust the business parameters and policy configuration. The data analysis results include data change trend charts, abnormal fluctuation detection results and key indicator warning information.

[0067] Furthermore, this embodiment also provides an online data processing system for a business intelligence service platform, including: a data acquisition module, which is used to collect the original data stream of the business intelligence service platform through a distributed acquisition gateway, pre-process the original data, and generate a normalized data set; a feature extraction module, which is used to extract spatiotemporal features from the normalized data set using a deep convolutional neural network model, and decompose the multi-scale components of the spatiotemporal features in combination with wavelet transform to form a feature combination matrix; a dynamic association graph construction module, which constructs a dynamic association graph for the feature combination matrix based on a graph attention network, and uses an edge probability calculation model to calculate the association strength between graph nodes to generate an association weight matrix; an establishment module, which is used to introduce an adaptive learning rate mechanism to the association weight matrix, dynamically update the weight parameters in combination with the exponential sliding average method, and establish a feature propagation network; a feedback module, which is used to input incremental business data into the feature propagation network, predict data change trends through an integrated learning algorithm, output data analysis results, and feed back to the business intelligence service platform

[0068] This embodiment also provides a computer device, which is applicable to an online data processing method for a business intelligence service platform, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the online data processing method for a business intelligence service platform proposed in the above embodiment.

[0069] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0070] In summary, the present invention accurately collects multi-source heterogeneous data through a distributed collection gateway, and generates a high-quality normalized data set after strict preprocessing; uses deep convolutional neural networks and wavelet transforms to extract data spatiotemporal features and enhance the depth and breadth of feature expression; constructs a dynamic association graph through a graph attention network to characterize the complex association relationship between data nodes, and introduces an adaptive learning rate mechanism to achieve real-time dynamic optimization of the feature propagation network; through an integrated learning algorithm, it can not only accurately predict data change trends, but also output multi-dimensional analysis results including change trend graphs, abnormal fluctuation detection, and key indicator warnings.

[0071] Example 2

[0072] Referring to Table 1, which is a second embodiment of the present invention, this embodiment provides an online data processing method for a business intelligence service platform. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0073] Specifically, in response to the data analysis needs of the platform under high-concurrency transaction scenarios, a business intelligence system based on a distributed architecture is deployed; by configuring 10 distributed acquisition gateway nodes, the interfaces of the order management system, customer relationship management system, inventory management system and financial accounting system are connected respectively; each gateway node is configured with a 4-core CPU and 16GB memory computing resource, and a Redis cluster is used as the data cache layer, with a single node processing capacity of 10,000 records per second. In the data preprocessing stage, 27 verification rules including field integrity, value range, format specification, etc. are set; by deploying the Spark distributed computing framework, parallel processing of data cleaning is realized, and the processing delay of raw data is controlled within 100ms.

[0074] Furthermore, in order to extract spatiotemporal features, a deep neural network with 5 convolutional layers is constructed. Each layer uses a 3×3 convolution kernel, a step size of 1, and the padding method is the same; the db4 wavelet basis function is selected for 4-layer wavelet decomposition, the time window size is set to 30 minutes, and the sliding step size is 5 minutes; in the feature fusion stage, the attention mechanism is used to adaptively weight features of different scales, and the feature dimension is set to 128 dimensions. The constructed graph attention network contains 3 multi-head attention layers, each layer has 8 attention heads, and the hidden layer dimension is 256.

[0075] Furthermore, to optimize the feature propagation network, the initial learning rate was set to 0.001, and the Adam optimizer was used for parameter update, with momentum parameters β1 = 0.9, β2 = 0.999. In the ensemble learning stage, a random forest model consisting of 100 decision trees and an XGBoost model consisting of 50 weak learners were constructed, and the model fusion adopted the Soft Voting strategy.

[0076] Specifically, as shown in Table 1, in terms of system performance, the data processing delay is reduced from 245ms of the traditional method to 98ms, an increase of 60.0%, thanks to the parallel processing architecture and optimized data caching strategy of the distributed acquisition gateway. The system throughput is increased from 5600 / second to 12500 / second, an increase of 123.2%, enhancing the system's ability to handle high-concurrency data. The average response time is reduced from 320ms to 180ms, with an optimization range of 43.8%, which greatly improves the real-time processing performance of the system. At the same time, resource utilization is increased from 65.8% to 88.9%, an increase of 35.1 percentage points, indicating that the present invention has obvious advantages in computing resource scheduling and utilization efficiency.

[0077] Table 1 Comparison between the present invention and the traditional method

[0078] Performance Indicators Traditional methods Method of the present invention Performance improvement Data processing delay (ms) 245 98 60.0％ Feature extraction accuracy (%) 85.3 94.8 11.1％ Prediction accuracy (%) 82.6 93.5 13.2％ System throughput (records / second) 5600 12500 123.2％ Resource utilization (%) 65.8 88.9 35.1％ Anomaly detection rate (%) 78.5 92.3 17.6％ Fault warning lead time (minutes) 15 35 133.3％ Average response time (ms) 320 180 43.8％

[0079] Furthermore, in terms of algorithm accuracy, the feature extraction accuracy rate increased from 85.3% to 94.8%, an increase of 11.1 percentage points, verifying the effectiveness of the feature extraction method of deep convolutional neural network combined with wavelet transform. The prediction accuracy increased from 82.6% to 93.5%, an increase of 13.2 percentage points, proving the superiority of the adaptive learning mechanism and integrated learning algorithm of the feature propagation network.

[0080] Furthermore, in terms of risk prevention and control, the present invention demonstrates stronger early warning capabilities. The anomaly detection rate increased from 78.5% to 92.3%, an increase of 17.6 percentage points, indicating that the dynamic association graph constructed by the present invention can more accurately identify data anomaly patterns. In particular, in terms of fault warning, the advance warning time was extended from 15 minutes to 35 minutes, an increase of 133.3%, providing more adequate response time for business risk prevention and control.

[0081] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An online data processing method for a business intelligence service platform, characterized in that: include, Collecting the original data stream of the business intelligence service platform through a distributed collection gateway, preprocessing the original data, and generating a normalized data set; Extracting spatiotemporal features from the normalized data set using a deep convolutional neural network model, and decomposing multi-scale components of the spatiotemporal features in combination with wavelet transform to form a feature combination matrix; Based on the graph attention network, a dynamic association graph is constructed for the feature combination matrix, and the edge probability calculation model is used to calculate the association strength between the graph nodes to generate an association weight matrix; An adaptive learning rate mechanism is introduced into the association weight matrix, and the weight parameters are dynamically updated in combination with the exponential sliding average method to establish a feature propagation network; The incremental business data is input into the feature propagation network, the data change trend is predicted through the integrated learning algorithm, the data analysis results are output, and fed back to the business intelligence service platform.

2. The online data processing method for a business intelligence service platform according to claim 1, characterized in that: The incremental business data is input into the feature propagation network, the data change trend is predicted through the integrated learning algorithm, the data analysis results are output, and the results are fed back to the business intelligence service platform, including: Preprocess the incremental business data and map the preprocessed incremental business data to the input layer nodes of the feature propagation network to extract and fuse feature information to generate high-dimensional feature data; Constructing several sub-models according to the high-dimensional feature data, wherein the sub-models include a gradient boosting tree model and a random forest model; The comprehensive prediction score is calculated based on the output results of the sub-models. The specific formula is as follows: Among them, S final is the comprehensive prediction score, is the gradient boosting tree model weight, is the random forest model weight, M1 is the number of gradient boosting trees, M2 is the number of random forests, G i is the prediction result of the i-th gradient boosting tree, yx is the confidence of the i-th gradient boosting tree, R j is the prediction result of the jth random forest, z j is the confidence of the jth random forest, δ is the bias correction term of the sub-model; The data analysis results are output according to the comprehensive prediction score and transmitted to the data display module of the business intelligence service platform to adjust the business parameters and strategy configuration, wherein the data analysis results include data change trend charts, abnormal fluctuation detection results and key indicator warning information.

3. The online data processing method for a business intelligence service platform according to claim 2, characterized in that: The comprehensive prediction score adopts a weighted voting method to fuse the prediction results of each sub-model; the method for establishing the feature propagation network is as follows: Setting an initial learning rate parameter for the associated weight matrix, and calculating an adaptive adjustment factor based on a weight gradient change trend, wherein the adaptive adjustment factor is dynamically adjusted as the weight gradient changes; In each round of iteration, the historical cumulative weights of the association weight matrix are calculated using an exponential moving average method; Updating the current weight parameter based on the product of the adaptive adjustment factor and the historical accumulated weight to form an optimized weight coefficient; The weight coefficients are input into the feature propagation layer to construct a feature propagation network with dynamic weight optimization capability.

4. The online data processing method for a business intelligence service platform according to claim 3, characterized in that: The method for generating the association weight matrix is: Constructing an initial node set in the graph attention network according to the feature combination matrix, and inputting the initial node set into the multi-head attention layer to calculate the attention coefficient between nodes, wherein the nodes in the initial node set represent feature vectors; Constructing a dynamic connection relationship of nodes according to the attention coefficient, and applying a nonlinear activation function to the dynamic connection relationship to generate a dynamic association graph; Using an edge probability calculation model to obtain the transition probability between node pairs in the dynamic association graph, wherein the transition probability represents the degree of association between nodes; The node pairs are weighted according to the transition probabilities to generate an association weight matrix.

5. The online data processing method for a business intelligence service platform according to claim 4, characterized in that: The method for forming the feature combination matrix is: The normalized data set is segmented according to the time series, a sliding time window is constructed, and the spatiotemporal features of the data in each time window are extracted through a deep convolutional neural network model; Selecting a wavelet basis function to perform discrete wavelet transform on the spatiotemporal characteristics, decomposing the characteristic signal into sub-components of different frequency bands, calculating the energy distribution characteristics of the sub-components of each frequency band, and extracting characteristic coefficients; Based on the feature coefficients, a multi-scale feature descriptor is constructed, and the multi-scale feature descriptor is fused with the transformed spatiotemporal features; The feature mapping algorithm is used to project features from different sources into the feature space to generate a feature combination matrix.

6. The online data processing method for a business intelligence service platform according to claim 5, characterized in that: The specific formula of the feature combination matrix is ​​as follows: Among them, M combined is the feature combination matrix, B is the feature dimension, γ i is the feature weight of the i-th dimension, F i is the eigenvalue of the i-th dimension, μ i is the feature mean of the i-th dimension, τ i is the standard deviation of the i-th dimension feature, λ is the smoothing factor, K is the number of sampling windows, ω k is the kth window weight, H k is the entropy value of the kth window, tanh(Δ) is the feature difference, E dist is the energy distribution characteristic value, and the specific formula is as follows: Among them, E dist is the energy distribution eigenvalue, J is the number of wavelet decomposition layers, β j is the energy weight coefficient of the jth layer, W j is the j-th layer wavelet coefficient, N is the number of frequency sampling points, P(f) is the power spectrum density at frequency f, ΔE is the energy difference between adjacent frequency bands, and S is the spectrum entropy.

7. The online data processing method for a business intelligence service platform according to claim 4 or 5, characterized in that: The deep convolutional neural network model includes a convolution layer, a pooling layer and a fully connected layer; the convolution layer uses convolution kernels of different scales to extract local features; the pooling layer reduces the feature dimension through the maximum pooling operation; the fully connected layer integrates the feature outputs of each layer; the method for forming the normalized data set is: Collect original data streams from various business system interfaces of the business intelligence service platform through distributed collection gateways; Based on the preset data verification rules, the original data stream is formatted and checked for integrity, and data records that do not conform to the specifications are removed; Using a data cleaning algorithm to process outliers, duplicate values, and missing values ​​in the original data stream; Perform field mapping and format conversion on the cleaned data according to data structure standards to generate standardized data records; The data records are classified and sorted according to business types to form a standardized data set, and efficient data access services are provided through a data indexing mechanism. At the same time, a data version control mechanism is set up to record the data processing history.

8. An online data processing system for a business intelligence service platform, based on the online data processing method for a business intelligence service platform according to any one of claims 1 to 7, characterized in that: Also includes, A data collection module is used to collect the original data stream of the business intelligence service platform through a distributed collection gateway, pre-process the original data, and generate a normalized data set; A feature extraction module, used to extract spatiotemporal features from the normalized data set using a deep convolutional neural network model, and decompose the multi-scale components of the spatiotemporal features in combination with wavelet transform to form a feature combination matrix; A dynamic association graph construction module constructs a dynamic association graph for the feature combination matrix based on a graph attention network, and uses an edge probability calculation model to calculate the association strength between graph nodes to generate an association weight matrix; Establishing a module for introducing an adaptive learning rate mechanism to the association weight matrix, dynamically updating the weight parameters in combination with an exponential sliding average method, and establishing a feature propagation network; The feedback module is used to input incremental business data into the feature propagation network, predict data change trends through an integrated learning algorithm, output data analysis results, and feed back to the business intelligence service platform.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the online data processing method for a business intelligence service platform described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the online data processing method for a business intelligence service platform described in any one of claims 1 to 7 are implemented.

Images