A big data-based business analysis method and system
By generating multi-source heterogeneous business time-series data and using independent component analysis algorithm to decouple equipment vibration characteristics and business indicators, a collaborative data flow is constructed, which solves the problems of latency and insufficient feature fusion in deep learning models and realizes real-time risk control and decision support.
Patent Information
- Application Number
- CN202511249004.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-03
- Publication Date
- 2025-11-21
- Estimated Expiration
- 2045-09-03
AI Technical Summary
In high-frequency trading and real-time risk control scenarios in finance, existing technologies suffer from insufficient real-time response capabilities due to training and inference delays in deep learning models, shallow fusion of multimodal data features, and poor adaptability of risk scores to sudden market fluctuations.
By acquiring user clickstream data, payment transaction data, and device electromagnetic interference signals, multi-source heterogeneous business time-series data is generated. Independent component analysis algorithms are used to decouple mixed business characteristics, construct collaborative data streams, and generate decision-making strategies in real time through a streaming processing engine.
It achieves synchronous collection and deep coupling of business operation data and physical equipment status data, eliminates data heterogeneity interference, provides clean business indicators to support real-time decision-making, and improves the efficiency of anomaly detection and risk response.
Smart Images

Figure CN120744477B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data technology, and in particular to a business analysis method and system based on big data. Background Technology
[0002] In high-frequency trading and real-time risk control scenarios in finance, systems need to simultaneously process heterogeneous data from multiple sources, including trading instruction streams, market data, and server operating status, and complete anomaly detection and risk response within sub-second timeframes. These scenarios require analytical methods capable of deep fusion and real-time processing of multi-dimensional data, while also considering the collaborative analysis of business logic and hardware status.
[0003] Current advanced solutions employ a deep learning-based multimodal data analysis framework. This framework uses a Long Short-Term Memory (LSTM) network to process transaction time-series data, combined with a convolutional neural network to analyze server performance metrics, and finally uses an attention mechanism to fuse the output of the two types of features. The system is equipped with a dynamic risk assessment model that generates a risk score based on the fused feature vector.
[0004] The training and inference processes of deep learning models have significant delays, making it difficult to meet the stringent real-time requirements of financial scenarios; feature fusion of multimodal data remains at the later decision-making level, failing to achieve deep coupling of underlying signals; and the generation of risk scores relies on static weights obtained from training on historical data, resulting in poor adaptability to sudden market fluctuations. Summary of the Invention
[0005] This application provides a business analysis method and system based on big data to solve the problem of low real-time response capability to abnormal fluctuations in high-concurrency business scenarios in the prior art.
[0006] Firstly, this application provides a business analysis method based on big data, including:
[0007] Acquire user clickstream data, payment transaction data, and device electromagnetic interference signals;
[0008] Based on the user clickstream data and the payment transaction data, multi-source heterogeneous business time-series data is generated.
[0009] The electromagnetic interference signal of the equipment is converted into the frequency domain characteristics of equipment vibration. Based on the frequency domain characteristics of equipment vibration and the multi-source heterogeneous service time series data, a collaborative data stream is generated. The collaborative data stream includes hybrid service characteristics.
[0010] The hybrid service characteristics are decoupled using the independent component analysis algorithm to obtain traffic density and bandwidth indicators;
[0011] The traffic density index and the bandwidth index are dynamically correlated to construct a service-independent feature matrix;
[0012] Based on the streaming processing engine, the business-independent feature matrix is converted into a business feature analysis graph, and the decision strategy corresponding to the business feature analysis graph is generated in real time.
[0013] Optionally, the step of decoupling the hybrid service characteristics using an independent component analysis algorithm to obtain traffic density and bandwidth metrics includes:
[0014] The separation factor is input into a pre-trained blind source separation model, and the traffic density signal source and bandwidth signal source are separated from the mixed service characteristics through the blind source separation model;
[0015] Extract the first fluctuation trajectory corresponding to the number of requests per minute from the traffic density signal source, analyze the first fluctuation trajectory, and obtain the traffic density index;
[0016] Extract the second fluctuation trajectory corresponding to the payment completion time from the bandwidth signal source, analyze the second fluctuation trajectory, and obtain the bandwidth index.
[0017] Optionally, the step of inputting the separation factor into a pre-trained blind source separation model, and separating the traffic density signal source and bandwidth signal source from the mixed service features through the blind source separation model, includes:
[0018] The flow density weight and bandwidth weight in the separation factor are used as separation criteria and set in the decoupling layer of the blind source separation model;
[0019] The decoupling layer is used to orthogonally project the hybrid service features to generate a feature vector space, which includes traffic density feature components and bandwidth feature components.
[0020] The separation hyperplane is determined based on the distribution difference between the flow density characteristic component and the bandwidth characteristic component;
[0021] Using the separating hyperplane, the feature vector space is divided into a first feature subspace and a second feature subspace;
[0022] Extract the flow density signal source from the first feature subspace and extract the bandwidth signal source from the second feature subspace.
[0023] Optionally, the step of using the separating hyperplane to divide the feature vector space into a first feature subspace and a second feature subspace includes:
[0024] Calculate the projection distance from each data point in the feature vector space to the separating hyperplane along the normal vector direction of the separating hyperplane;
[0025] The boundary region is determined based on the sign of the projected distance;
[0026] Based on the boundary region, the feature vector space is divided into two orthogonal subspaces;
[0027] The orthogonal subspace containing the clustered region of the flow density feature component is defined as the first feature subspace;
[0028] The orthogonal subspace containing the region where the bandwidth feature components are clustered is defined as the second feature subspace.
[0029] Optionally, generating a collaborative data stream based on the device vibration frequency domain characteristics and the multi-source heterogeneous service time-series data includes:
[0030] The vibration frequency domain characteristics of the equipment are superimposed with the multi-source heterogeneous service time-series data according to the sampling time points to generate superimposed waveform data.
[0031] Through a signal injection mechanism, the superimposed waveform data is re-encapsulated into a service data packet according to the transmission protocol of the service data stream;
[0032] Align the physical layer feature timestamps in the service data packets with the service data timestamps using execution timestamps;
[0033] Based on the aligned business data packets, a collaborative data stream containing mixed business characteristics is output.
[0034] Optionally, the step of dynamically associating the traffic density index and the bandwidth index to construct a service-independent feature matrix includes:
[0035] Establish a sliding time window, and within the sliding time window, calculate the fluctuation coupling degree between the flow density index and the bandwidth index;
[0036] Based on the aforementioned fluctuation coupling degree, the flow density index value and bandwidth index value are weighted respectively;
[0037] The weighted traffic density and bandwidth values are arranged in time series to generate a business-independent feature matrix.
[0038] Optionally, the step of converting the business-independent feature matrix into a business feature analysis graph based on a streaming processing engine, and generating the decision strategy corresponding to the business feature analysis graph in real time, includes:
[0039] Create corresponding virtual data channels for the business-independent feature matrix in the streaming processing engine;
[0040] In the virtual data channel, the business-independent feature matrix is decomposed into multiple data blocks, and each data block is mapped to an analysis node;
[0041] Calculate the association strength between the analysis nodes, and establish the connection relationship between the analysis nodes based on the association strength;
[0042] Based on the aforementioned connection relationships, a business feature analysis graph is constructed;
[0043] Based on the topological characteristics of the business feature analysis graph, a corresponding decision-making strategy is generated.
[0044] Secondly, this application provides a big data-based business analysis system, including:
[0045] The acquisition module is used to acquire user clickstream data, payment transaction data, and device electromagnetic interference signals.
[0046] The first generation module is used to generate multi-source heterogeneous business time-series data based on the user clickstream data and the payment transaction data;
[0047] The conversion module is used to convert the electromagnetic interference signal of the equipment into the frequency domain characteristics of the equipment vibration, and generate a collaborative data stream based on the frequency domain characteristics of the equipment vibration and the multi-source heterogeneous service time series data. The collaborative data stream includes mixed service characteristics.
[0048] The decoupling module is used to decouple the hybrid service characteristics using an independent component analysis algorithm to obtain traffic density and bandwidth indicators.
[0049] A construction module is used to dynamically correlate the traffic density index and the bandwidth index to construct a service-independent feature matrix;
[0050] The second generation module is used to convert the business-independent feature matrix into a business feature analysis graph based on the streaming processing engine, and generate the decision strategy corresponding to the business feature analysis graph in real time.
[0051] Thirdly, this application provides a computing device including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a big data-based business analysis method as described in any of the first aspects.
[0052] Fourthly, this application provides a computer storage medium storing computer program instructions thereon, which, when executed by a processor, implement a big data-based business analysis method as described in any one of the first aspects.
[0053] The technical solution provided in this application has the following beneficial effects:
[0054] This application achieves synchronous acquisition of operational data and physical equipment status data, providing a complete data foundation for subsequent multi-dimensional analysis. It unifies operational data from different sources into a standardized time-series format, eliminating the interference of data heterogeneity on analysis. It transforms physical layer signals into quantifiable frequency domain parameters, revealing the potential correlation between equipment operating status and operational data. It establishes a deep coupling relationship between operational data and equipment characteristics, forming a data carrier containing hybrid features. It separates independent operational indicators from complex data, eliminating cross-interference between features. Through dynamic correlation of indicators, it forms a structured data representation, preserving the time-dimensional characteristics of operational processes. It achieves an intuitive and visual representation of operational status, supporting real-time decision-making.
[0055] Furthermore, this application also uses a separation factor-driven blind source separation model to accurately separate the traffic density signal source and bandwidth signal source from the mixed business characteristics, and then extracts the request number fluctuation trajectory as the traffic density indicator and the payment time fluctuation trajectory as the bandwidth indicator.
[0056] Furthermore, it achieves essential decoupling of business characteristics, keeping the two core indicators of traffic load and transmission capacity in an independent and analyzable state, providing pure feature input for subsequent accurate decision-making.
[0057] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description
[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0059] Figure 1 A flowchart illustrating a big data-based business analysis method provided in this application embodiment;
[0060] Figure 2 A schematic diagram of the structure of a big data-based business analysis system provided in this application embodiment;
[0061] Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation
[0062] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0063] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.
[0064] Current deep learning-based multimodal business analysis methods suffer from three key limitations: First, the model inference latency fundamentally contradicts the real-time requirements of high-concurrency scenarios, leading to delayed risk response. Second, the shallow feature fusion of business data and physical layer signals struggles to capture the intrinsic correlation between equipment vibration and business fluctuations, limiting anomaly detection sensitivity. Finally, static weighting mechanisms cannot adapt to sudden market fluctuations, causing risk scoring to become disconnected from actual business conditions. These shortcomings stem from the dual neglect of data coupling depth and timeliness in existing technologies, necessitating the construction of a deep collaborative analysis framework for business and physical signals.
[0065] To address the aforementioned issues, this application proposes a big data-based business analysis method. This method constructs a collaborative data stream containing hybrid features through cross-domain fusion of equipment electromagnetic interference signals and business time-series data. Innovatively, this method employs independent component analysis (ICA) to decouple physical vibration characteristics from business indicators, achieving pure extraction of traffic density and bandwidth metrics, and constructing a business feature matrix through dynamic correlation. Unlike the batch processing mode of existing technologies, this solution uses a streaming engine to convert feature maps in real time, directly compensating for business data based on physical layer signal drift, simultaneously improving the timeliness and accuracy of analysis. This method establishes a breakthrough deep coupling relationship between equipment status and business indicators, enabling millisecond-level anomaly detection and adaptive decision-making, completely solving the three major shortcomings of existing technologies: slow response, fragmented features, and insufficient adaptability.
[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0067] Figure 1 A flowchart of a big data-based business analysis method provided in this application embodiment is shown below. Figure 1 As shown, the method includes:
[0068] Step 101: Obtain user clickstream data, payment transaction data, and device electromagnetic interference signals.
[0069] In step 101, user clickstream data represents records of user browsing and clicking behavior on the platform, including click time, page location, etc. Payment transaction data represents records of successful transaction order information, including amount, payment time, etc. Device electromagnetic interference signal represents the electromagnetic radiation signal generated by the server during operation, reflecting the device's working status.
[0070] In this embodiment, the system collects data from three channels simultaneously: user click behavior logs obtained from front-end tracking points, transaction records obtained from the payment gateway interface, and electromagnetic signals around the server collected by electromagnetic sensors. The three types of data are aligned with a unified time base to ensure the consistency of subsequent processing timing.
[0071] For example, during a flash sale on e-commerce platform A, the system collects user click records on product pages, including click timestamps and product numbers; it simultaneously obtains order numbers, payment amounts, and payment success times returned by the payment system; and it captures electromagnetic signal strength data at a fixed sampling frequency using electromagnetic sensors deployed in the server rack. All three types of data are tagged with timestamps at the time of collection, and the time synchronization error is controlled within an acceptable range.
[0072] Step 102: Generate multi-source heterogeneous business time-series data based on the user clickstream data and the payment transaction data.
[0073] In step 102, the multi-source heterogeneous business time-series data representation converts business data from different sources into a unified time-series format.
[0074] In this embodiment of the application, the request volume of user clickstream data is aggregated and calculated by minute, and the transaction amount of payment flow data is summarized by minute. The two types of data are aligned and merged by timestamp to form a time-series data table containing two dimensions: the request volume per minute and the transaction amount.
[0075] For example, the click data collected in step 101 is aggregated every minute to calculate the number of clicks on the product page per minute; at the same time, payment data is summarized every minute to calculate the total amount of successful transactions per minute. After aligning the two types of data with timestamps, a time-series data table containing three fields is generated.
[0076] Step 103: Convert the electromagnetic interference signal of the equipment into the frequency domain characteristics of the equipment vibration, and generate a collaborative data stream based on the frequency domain characteristics of the equipment vibration and the multi-source heterogeneous service time series data. The collaborative data stream includes hybrid service characteristics.
[0077] In step 103, the equipment vibration frequency domain characteristics represent the equipment vibration frequency distribution characteristics obtained from the electromagnetic signal conversion. The collaborative data stream represents the fused data carrier of business data and equipment characteristics. The hybrid business characteristics represent comprehensive characteristics including business indicators and equipment status.
[0078] In this embodiment, the electromagnetic signal is subjected to spectrum analysis to extract the energy distribution of the main vibration frequency band as the device feature; this feature is matched with the service time series data at time points, and a new data stream containing service and device features is generated by weighted fusion.
[0079] For example, the electromagnetic signal is subjected to a fast Fourier transform to obtain a spectrum, and the three frequency bands with the highest energy are selected as features. These feature values are then combined with the service data at the corresponding time, and the service data is multiplied by an adjustment coefficient based on the device features to generate a fused data record.
[0080] Step 104: Decouple the hybrid service characteristics using the independent component analysis algorithm to obtain traffic density and bandwidth indicators.
[0081] In step 104, the traffic density metric represents a quantitative indicator reflecting the pressure of service requests. The bandwidth metric represents a quantitative indicator reflecting the system's processing capacity.
[0082] In this embodiment, the independent component analysis algorithm is used to process mixed features. Separation parameters are set to project the data into a new space, and projection directions that make the features independent of each other are found. Request volume-related components are extracted as traffic density indicators, and processing time-related components are extracted as bandwidth indicators.
[0083] For example, a separation model can be established for the fused data, and the optimal projection direction can be found through iterative calculation. The data can be decomposed into two parts: one part mainly contains information on changes in request volume as a traffic density indicator, and the other part mainly contains information on processing time as a bandwidth indicator.
[0084] Step 105: Dynamically correlate the traffic density index and the bandwidth index to construct a service-independent feature matrix.
[0085] In step 105, the business-independent feature matrix represents the matrix representation of the correlation results of the indicators.
[0086] In this embodiment, a sliding time window is established, and the covariance relationship between the traffic density index and the bandwidth index is calculated within the window. The index values are dynamically weighted according to the strength of the correlation, and the weighted results are arranged in chronological order to form a feature matrix.
[0087] For example, take a 5-minute sliding window, calculate the covariance between the number of requests per minute and the processing time, use the covariance value as a weight to adjust the original indicator value, and arrange the adjusted values into a matrix in chronological order, with each row representing a time point and each column representing an indicator.
[0088] Step 106: Based on the streaming processing engine, the business-independent feature matrix is converted into a business feature analysis graph, and the decision strategy corresponding to the business feature analysis graph is generated in real time.
[0089] In step 106, the business feature analysis graph represents a graphical representation of the business status. The decision strategy represents the operation instructions generated based on the analysis results.
[0090] In this embodiment, the feature matrix is input into the stream processing engine, the matrix rows are converted into graph nodes, the similarity between nodes is calculated to establish edge relationships, abnormal patterns are identified based on graph structure features, and corresponding expansion or rate limiting instructions are generated.
[0091] For example, each row of the feature matrix is treated as a node, and the similarity of request volume and processing time between nodes is calculated. Nodes with high similarity are connected to form a business status relationship graph. When an abnormal connection pattern is detected, an automatic adjustment strategy is triggered.
[0092] This method achieves collaborative monitoring of business status and equipment operation through synchronous collection and deep fusion of multi-source data; it obtains pure business indicators by using feature decoupling technology to avoid cross-interference; it ensures the real-time and accuracy of analysis results through dynamic correlation and streaming processing; and the resulting visual analysis chart provides intuitive basis for decision-making, improving the efficiency of business anomaly detection and handling in high-concurrency scenarios.
[0093] To address the deep decoupling issue between business data and device status characteristics, and to further improve the accuracy of indicator extraction, in some embodiments, step 104: using independent component analysis (ICA) to decouple the mixed business characteristics to obtain traffic density and bandwidth indicators includes:
[0094] Step 201: Input the separation factor into the pre-trained blind source separation model, and separate the traffic density signal source and bandwidth signal source from the mixed service characteristics through the blind source separation model.
[0095] In step 201, the separation factor is a pre-defined combination of weighting parameters, comprising two parts: a traffic density weight value and a bandwidth weight value. Its design is based on the statistical distribution differences between the traffic density component and the bandwidth component in the mixed service characteristics. The specific values are determined by calculating the variance ratio and correlation coefficient of the two types of components. The core meaning of this factor is to guide the blind source separation model in distinguishing and extracting the decoupling criteria of the traffic density signal source and the bandwidth signal source. The blind source separation model refers to a computational model capable of separating independent components from a mixed signal. Its decoupling layer achieves feature space reconstruction through orthogonal transformation. The traffic density signal source refers to the data component that, after separation, mainly reflects the service request pressure. The bandwidth signal source refers to the data component that, after separation, mainly reflects the system processing capacity.
[0096] In this embodiment, the pre-calculated separation factor parameters are configured into the model decoupling layer. After receiving the mixed service feature input, the model projects the data onto a new feature space through orthogonal transformation. In this space, the optimal segmentation surface is found according to the direction indicated by the separation factor, and the mixed data is separated into two independent data sets, which are output as traffic density signal source and bandwidth signal source, respectively.
[0097] Step 202: Extract the first fluctuation trajectory corresponding to the number of requests per minute from the traffic density signal source, analyze the first fluctuation trajectory, and obtain the traffic density index.
[0098] In step 202, the first fluctuation trajectory refers to the curve shape of the number of requests changing over time in the flow density signal source. The flow density index refers to the quantitative evaluation value obtained by analyzing this curve.
[0099] In this embodiment of the application, the change curve of the number of requests per minute is extracted from the separated traffic density signal source, and the peak and trough characteristics and change trends of the curve are analyzed. A comprehensive evaluation value reflecting the business pressure status is calculated by a specific algorithm as a traffic density index.
[0100] Step 203: Extract the second fluctuation trajectory corresponding to the payment completion time from the bandwidth signal source, analyze the second fluctuation trajectory, and obtain the bandwidth index.
[0101] In step 203, the second fluctuation trajectory refers to the curve shape of the payment completion time changing over time in the bandwidth signal source. The bandwidth index is a quantitative evaluation value obtained by analyzing this curve.
[0102] In this embodiment of the application, the change curve of payment completion time is extracted from the separated bandwidth signal source, and the time distribution characteristics, abnormal fluctuation points and other elements of the curve are analyzed. A comprehensive evaluation value reflecting the system's processing capability is calculated as a bandwidth index through a specific algorithm.
[0103] Here is a specific example:
[0104] During a flash sale on e-commerce platform A, the system inputs the generated fused data records into a pre-trained blind source separation model. The model receives a separation factor consisting of a traffic density weight of 0.7 and a bandwidth weight of 0.3. These two weights are calculated by taking the variance ratio of the number of requests to the payment completion time in historical data. The formula for calculating the variance ratio is as follows: ,in Indicates the variance ratio. The variance of the number of requests. The model first projects the fused data onto a new coordinate system and determines the optimal separation direction by calculating the eigenvectors of the covariance matrix. The data is then decomposed into two parts: the first part mainly contains fluctuating data with requests per minute ranging from 1200 to 1500, serving as a traffic density signal source; the second part mainly contains fluctuating data with payment completion times ranging from 200 to 400 milliseconds, serving as a bandwidth signal source. From the traffic density signal source, the model extracts the minute-by-minute variation curve of the number of requests and calculates the average fluctuation amplitude of the curve as the traffic density index, specifically taking the average of the differences between the maximum and minimum number of requests per minute. From the bandwidth signal source, the model extracts the minute-by-minute variation curve of the payment completion time and calculates the fluctuation frequency of the curve as the bandwidth index, specifically counting the number of times the payment completion time exceeds 300 milliseconds per minute. The resulting traffic density index reflects system load pressure, and the bandwidth index reflects transaction processing efficiency. Both indicators are used for subsequent business status analysis and decision-making.
[0105] In this embodiment, the method achieves deep decoupling of service characteristics and device characteristics through precise signal separation technology, resulting in purer and more reliable indicator data. The dynamic weighting mechanism ensures that the decoupling process can adapt to the characteristics of different business scenarios. The final traffic density and bandwidth indicators provide an accurate basis for system status assessment and effectively support subsequent business decisions.
[0106] To address the issue of accurate signal source separation in mixed service characteristics and further improve decoupling accuracy, in some embodiments, step 201: inputting the separation factor into a pre-trained blind source separation model, and using the blind source separation model to separate the traffic density signal source and bandwidth signal source from the mixed service characteristics, includes:
[0107] Step 301: Set the flow density weight and bandwidth weight in the separation factor as separation criteria in the decoupling layer of the blind source separation model.
[0108] In step 301, traffic density weight is a parameter reflecting the importance of service request pressure; a larger value indicates a higher proportion of that feature in the separation process. Bandwidth weight is a parameter reflecting the importance of system processing capacity; its value determines the priority of the corresponding feature during separation. The decoupling layer is a network layer in the blind source separation model specifically responsible for feature decomposition, which distinguishes different features by adjusting internal parameters.
[0109] In this embodiment, the calculated traffic density weight and bandwidth weight parameters are input into the configuration unit of the decoupling layer. This unit adjusts the parameter settings of the internal projection matrix according to the weight ratio, so that the model can distinguish different features according to the preset importance in subsequent processing.
[0110] Step 302: Through the decoupling layer, orthogonally project the hybrid service features to generate a feature vector space, which includes traffic density feature components and bandwidth feature components.
[0111] In step 302, orthogonal projection refers to a spatial transformation method that preserves the original relationships between data. The eigenvector space refers to the mathematical space in which the data resides after projection, with each dimension representing a characteristic component. The traffic density characteristic component refers to the mathematical representation of the data characterizing the request load in the mixed service characteristics in the eigenvector space after orthogonal projection; the bandwidth characteristic component refers to the mathematical representation of the data characterizing the transmission capacity in the mixed service characteristics in the eigenvector space after orthogonal projection. Both are transformed from the original mixed service characteristics through orthogonal projection processing in the decoupling layer. The traffic density signal source is a set of pure request load signals extracted from the first characteristic subspace, its data basis being the distribution of the traffic density characteristic components in that subspace. The bandwidth signal source is a set of pure transmission capacity signals extracted from the second characteristic subspace, its data basis being the distribution of the bandwidth characteristic components in that subspace. The characteristic component is the representation of the signal source in the mathematical space, and the signal source is the physical entity obtained after spatial partitioning of the characteristic component. In this embodiment of the application, the decoupling layer with configured parameters receives mixed service feature input and transforms the original data into a new coordinate system through matrix operations. In the new coordinate system, each data point is represented by a set of feature vectors, which mainly include two types of components: vectors reflecting request pressure and vectors reflecting processing capacity.
[0112] Step 303: Determine the separating hyperplane based on the distribution difference between the flow density characteristic component and the bandwidth characteristic component.
[0113] In step 303, distribution difference refers to the degree of aggregation and relative positional relationship of different feature components in space. A separating hyperplane is a mathematical plane that can best distinguish between two types of feature components.
[0114] In this embodiment, the aggregation regions and distribution ranges of two types of feature components in the feature vector space are analyzed, their center point distance and distribution dispersion are calculated, and a plane equation that can separate the two types of features to the greatest extent is found. This plane is the separating hyperplane.
[0115] Step 304: Using the separating hyperplane, the feature vector space is divided into a first feature subspace and a second feature subspace.
[0116] In step 304, the first feature subspace refers to the spatial region that mainly contains the flow density feature component. The second feature subspace refers to the spatial region that mainly contains the bandwidth feature component.
[0117] In this embodiment of the application, based on the determined separating hyperplane equation, the positional relationship of each point in space relative to the plane is calculated, and points located on one side of the plane are assigned to the first feature subspace, and points on the other side are assigned to the second feature subspace, thereby realizing the division of the spatial region.
[0118] Step 305: Extract the flow density signal source from the first feature subspace and extract the bandwidth signal source from the second feature subspace.
[0119] In step 305, the traffic density signal source refers to the pure service pressure characteristic data extracted from the first feature subspace. The bandwidth signal source refers to the pure system processing capacity characteristic data extracted from the second feature subspace.
[0120] In this embodiment of the application, data is extracted from the two divided feature subspaces respectively. The data points in the first feature subspace are converted back to the original feature form as the flow density signal source, and the data points in the second feature subspace are converted back to the original feature form as the bandwidth signal source.
[0121] Here is a specific example:
[0122] In a promotional campaign on e-commerce platform C, the system configures prepared separation factor parameters into the blind source separation model. The traffic density weight (0.65) and bandwidth weight (0.35) are calculated by analyzing historical promotional data. Specifically, the calculation method involves averaging the ratio of the fluctuation range of request counts to the payment delay duration across the past three promotional campaigns. After receiving mixed data containing the number of requests per minute and payment time, the model first projects the data onto a two-dimensional feature space through orthogonal transformation. The horizontal axis primarily reflects the request volume characteristics, and the vertical axis primarily reflects the processing time characteristics. The model then calculates the distance between the distribution centers of the two types of feature components. Determine the optimal dividing line, where This represents the distance between the distribution centers of the two types of feature components. The distribution center of the request volume characteristics, To determine the distribution center of the duration feature; a straight line is selected based on the calculation results. As a separating hyperplane, the feature space is divided into two regions; the data points in the upper left region mainly correspond to feature combinations with request counts of 1250-1550 times / minute and payment times of 180-350 milliseconds, and are used as traffic density signal sources after conversion back to the original data; the data points in the lower right region mainly correspond to feature combinations with request counts of 900-1200 times / minute and payment times of 380-450 milliseconds, and are used as bandwidth signal sources after conversion.
[0123] In this embodiment, the method achieves efficient decoupling of mixed business features through weight-guided spatial projection and precise partitioning; the separated signal sources maintain the temporal characteristics and physical meaning of the original data; the feature separation method based on mathematical space avoids the bias caused by subjective judgment and provides a reliable data foundation for subsequent analysis.
[0124] To further improve the accuracy of feature space segmentation, in some embodiments, step 304: using the separating hyperplane to segment the feature vector space into a first feature subspace and a second feature subspace includes:
[0125] Step 401: Calculate the projection distance from each data point in the feature vector space to the separating hyperplane along the normal vector direction of the separating hyperplane.
[0126] In step 401, the normal vector direction refers to the perpendicular direction vector of the separating hyperplane, derived from the optimal separating direction calculated based on the distribution difference between the flow density feature component and the bandwidth feature component, and is used to define the reference axis for spatial segmentation. Data points include all projection points of the flow density feature component and the bandwidth feature component in the feature vector space, and are the mathematical representation of the mixed service features after orthogonal projection. The projection distance refers to the perpendicular distance of the data point along the normal vector direction to the hyperplane, and its magnitude reflects the distance between the data point and the hyperplane.
[0127] In this embodiment of the application, based on the determined separating hyperplane equation, the vertical distance from each data point in the feature space to the plane along a specific direction is calculated, and this distance value will be used as the basis for subsequent spatial partitioning.
[0128] Step 402: Determine the boundary region based on the sign of the projection distance.
[0129] In step 402, the boundary region refers to the transition region where the projected distance value is close to zero, which contains data points that may have both types of characteristics at the same time; positive or negative refers to the sign of the projected distance value, which is used to initially determine the category to which the data points belong.
[0130] In this embodiment, the projected distance values of all data points are judged by sign, and positive and negative values are classified separately. At the same time, a narrow region around the hyperplane is defined as a boundary buffer, and the data points in this region need to be further analyzed and confirmed.
[0131] Step 403: Based on the boundary region, divide the feature vector space into two orthogonal subspaces.
[0132] In step 403, an orthogonal subspace refers to a spatial region that is perpendicular to each other and does not overlap. Each subspace mainly contains data points of a certain type of feature.
[0133] In this embodiment of the application, the feature space is divided into two non-interfering regions by using the boundary region as the boundary, so as to ensure that the data points in each region have similar feature attributes.
[0134] Step 404: Determine the orthogonal subspace containing the clustered region of the flow density feature components in the two orthogonal subspaces as the first feature subspace.
[0135] In step 404, the flow density feature component clustering region refers to the continuous spatial range in the feature vector space where the density of the projection points of the flow density feature components is higher than that of the bandwidth feature components. It is obtained by calculating the kernel density of the projection points and estimating and detecting the density peak region.
[0136] In this embodiment of the application, the distribution of data points within the divided subspace is analyzed, and the region where the request volume feature points are concentrated is identified. This region is then determined as the first feature subspace that mainly reflects the business pressure.
[0137] Step 405: Determine the orthogonal subspace containing the bandwidth feature component cluster region as the second feature subspace.
[0138] In step 405, the bandwidth feature component clustering region refers to the continuous spatial range in the feature vector space where the projection point density of the bandwidth feature component is higher than that of the flow density feature component, which is automatically identified through spectral energy distribution analysis and spatial clustering algorithm.
[0139] In this embodiment of the application, the distribution of data points in another subspace is analyzed, and the region in which the processing time feature points are concentrated is identified. This region is determined as the second feature subspace that mainly reflects the system's processing capability.
[0140] Here is a specific example:
[0141] During a limited-time flash sale on e-commerce platform E, the system was based on a pre-defined separating hyperplane. To perform feature space segmentation, the first step is to calculate the projection distance from each data point in the feature space to the hyperplane. The formula for calculating the projection distance is as follows: , This represents the projected distance from the data point to the separating hyperplane. Analysis revealed that the projected distances of the request volume feature data points are mainly concentrated in the range of 0.3 to 1.2, while the projected distances of the processing time feature data points are mainly in the range of -0.4 to -1.1. The region with an absolute value of the projected distance less than 0.15 is defined as the boundary region, which contains approximately 5% of the transitional data points. Using the boundary region as the boundary, the feature space is divided into two subspaces: the left subspace contains 85% of the request volume feature data points, and these points are mainly clustered in the range of x-coordinate 1.1 to 1.5 and y-coordinate 0.8 to 1.2, thus it is determined as the first feature subspace; the right subspace contains 82% of the processing time feature data points, and these points are mainly clustered in the range of x-coordinate 0.7 to 1.1 and y-coordinate 1.3 to 1.7, thus it is determined as the second feature subspace.
[0142] In this embodiment, the method achieves clear separation of business features through precise spatial projection and region division; the setting of boundary regions avoids misclassification of edge data; the two feature subspaces obtained in the end completely retain the core features of the original data, providing a high-quality data foundation for subsequent analysis.
[0143] To further improve the fusion effect of business data and equipment status data, in some embodiments, step 103: generating a collaborative data stream based on the equipment vibration frequency domain characteristics and the multi-source heterogeneous business time-series data includes:
[0144] Step 501: The vibration frequency domain characteristics of the equipment and the time-series data of the multi-source heterogeneous services are superimposed on each other according to the sampling time points to generate superimposed waveform data.
[0145] In step 501, the sampling time point refers to the same time reference extracted from the equipment vibration frequency domain characteristics and multi-source heterogeneous service time-series data. It originates from the preset synchronization clock signal of the original acquisition system for both types of data, ensuring time axis alignment during waveform overlay. Waveform overlay refers to a processing method that fuses data from different sources after aligning them according to time. The overlaid waveform data refers to the fused data waveform, which simultaneously contains service characteristic and equipment characteristic information.
[0146] In this embodiment, the frequency domain characteristic values of equipment vibration are mapped to the same time points as the service time series data. The two types of data are merged into new waveform data through weighted calculation to ensure that the data at each time point simultaneously reflects the service status and equipment status.
[0147] Step 502: Through a signal injection mechanism, the superimposed waveform data is re-encapsulated into a service data packet according to the transmission protocol of the service data stream.
[0148] In step 502, the signal injection mechanism refers to the technique of embedding processed data into the business data stream according to a specific format. The transmission protocol of the business data stream refers to the data encapsulation rules followed when payment transaction data and user clickstream data interact between the acquisition end and the processing end, originating from the communication standards preset by the business system (such as ISO 8583 in the financial field or the HTTP / HTTPS data format of e-commerce platforms). A business data packet refers to a data unit encapsulated according to the communication protocol, containing valid data and protocol header information.
[0149] In this embodiment of the application, the superimposed waveform data is reorganized according to the data format specified by the business system, and necessary timestamps and data type identifiers are added to form a standardized data packet that meets the transmission requirements.
[0150] Step 503: Align the physical layer feature timestamp in the service data packet with the service data timestamp using the execution timestamp.
[0151] In step 503, the service data packet simultaneously contains both physical layer feature timestamps (from the acquisition time of the device vibration frequency domain features) and service data timestamps (from the generation time of the user click stream / payment transaction record), which exist as independent fields in the same data packet. Timestamp alignment refers to the process of adjusting the timestamps of data from different sources to achieve time synchronization.
[0152] In this embodiment of the application, the timestamps of device features and service features in the data packet are checked, and timestamps with slight differences are calibrated to ensure that the two types of data are completely synchronized in the time dimension.
[0153] Step 504: Based on the aligned service data packets, output a collaborative data stream containing mixed service characteristics.
[0154] In this embodiment of the application, the data packets that have completed time calibration are arranged in chronological order, and after removing redundant information, they are output as a continuous data stream. This data stream retains all the key information of the original business data and device characteristics.
[0155] Here is a specific example:
[0156] During a limited-time flash sale on e-commerce platform G, the system fused the vibration characteristic values (0.7, 1.1, and 0.8) of three main frequency bands collected by electromagnetic sensors with the data of 1500 page clicks per minute and a total transaction amount of 60,000 yuan. The specific processing steps were as follows: First, the vibration characteristic values were normalized to fall within the range of 0 to 1, where the normalization formula was... ,in This represents the normalized vibration eigenvalues. Indicates the original vibration value. and The minimum and maximum vibration values in the historical data are represented respectively. Then, the normalized vibration value is multiplied by an adjustment factor of 0.015 and added to the click volume, and then multiplied by 0.00012 and added to the transaction amount to generate the superimposed waveform data. Next, according to the platform's custom data transmission protocol, a protocol header is added to each superimposed data, containing information such as timestamp, data type identifier, and data length. It was found that the vibration data timestamp is on average 15 milliseconds ahead of the business data, so all vibration feature timestamps are calibrated with a 15-millisecond delay. The final output is a collaborative data stream containing the calibration timestamp, adjusted click volume, adjusted transaction amount, and original vibration feature values, generating one complete record per minute. The adjusted click volume = original click volume + 0.015 × normalized vibration value, and the adjusted transaction amount = original transaction amount + 0.00012 × normalized vibration value. This data will be used for subsequent business status analysis and anomaly detection.
[0157] In this embodiment, the method achieves deep coupling between business data and device status data through precise time alignment and numerical fusion; the generated collaborative data stream retains the characteristics of the original data while establishing the correlation between business and device; the standardized data format ensures that subsequent processing modules can efficiently parse and use these data, providing a more comprehensive data foundation for business status analysis.
[0158] To further improve the dynamic correlation analysis capability between business metrics, in some embodiments, step 105: dynamically correlating the traffic density metric and the bandwidth metric to construct a business-independent feature matrix includes:
[0159] Step 601: Establish a sliding time window, and within the sliding time window, calculate the fluctuation coupling degree between the flow density index and the bandwidth index.
[0160] In step 601, the sliding time window refers to a data analysis interval that moves along the time axis, used to capture short-term change characteristics of indicators. The volatility coupling degree refers to the degree of coordinated change between two indicators within the time window, reflecting their dynamic correlation.
[0161] In this embodiment, a fixed-length time window is set, and the coordinated changes of traffic density and bandwidth indicators are calculated within the window. A specific correlation algorithm is used to obtain the coupling degree between the two. The larger the value, the stronger the linkage between the indicators.
[0162] Step 602: Based on the fluctuation coupling degree, weight the flow density index value and the bandwidth index value respectively.
[0163] In step 602, weighting refers to the process of adjusting the numerical weights of indicators based on the strength of the relationship between them. The traffic density indicator value and the bandwidth indicator value are derived from the intensity change trajectory of the traffic density signal source and the spectral width change trajectory of the bandwidth signal source, respectively, extracted by decoupling hybrid service characteristics. They are time-series data points after quantization. The traffic density indicator value is a specific numerical instance of the traffic density indicator within a sliding time window, and the bandwidth indicator value is a specific numerical instance of the bandwidth indicator within a sliding time window; both are quantitative representations of the indicators in the time dimension.
[0164] In this embodiment, the calculated fluctuation coupling degree is used as an adjustment coefficient to proportionally adjust the flow density index value and bandwidth index value within the window, so that the adjusted values can reflect the actual correlation strength between the indices.
[0165] Step 603: Arrange the weighted traffic density index value and the weighted bandwidth index value according to the time series to generate a service-independent feature matrix.
[0166] In this embodiment, the weighted index values calculated for each time window are arranged in chronological order, with the traffic density index as the first column and the bandwidth index as the second column, to construct a structured data matrix for easy subsequent analysis and processing.
[0167] Here is a specific example:
[0168] In a limited-time flash sale event on e-commerce platform I, the system uses a 5-minute sliding window to dynamically correlate traffic density and bandwidth metrics. The specific implementation process is as follows: First, within the first time window, data sequences of request counts and payment completion times per minute are acquired. The request count sequence is 1300, 1400, 1500, 1450, and 1350 times / minute, and the payment completion time sequence is 220, 240, 260, 250, and 230 milliseconds. The covariance of the two sequences is then calculated using the formula... ,in express and covariance value, Represents a sequence of request counts. Represents the payment time series. and Let represent the means of the two sequences, respectively. The number of data points in the window is 5. Represents the first in the sequence of request counts One element, The first in the payment completion time series The covariance value is calculated to be 1250 from the elements; this covariance value is then converted into fluctuation coupling degree using the formula... ,in and The standard deviations of the two sequences are represented, and the coupling degree is calculated to be 0.68. This coupling degree is used to weight the request count values (original values multiplied by 0.68) to obtain adjusted request count values of 884, 952, 1020, 986, and 918 times / minute. The payment completion time values are then weighted by multiplying by 0.32 to obtain adjusted payment completion times of 70.4, 76.8, 83.2, 80.0, and 73.6 milliseconds. These adjusted values are arranged in chronological order to form the first row of the business-independent feature matrix. After the window slides for one minute, the data in the new window is processed using the same method to generate the second row of the matrix. This process continues until all monitoring periods are covered, ultimately constructing a complete business-independent feature matrix. Each row of data reflects the dynamic relationship between request volume and processing time, providing data support for subsequent business status analysis and decision-making.
[0169] In this embodiment, the method accurately captures the real-time correlation between business indicators through dynamic analysis of sliding windows; the weighted adjustment based on coupling degree makes the feature matrix better reflect the actual operating status of the system; and the structured matrix form provides a clear and standardized data foundation for subsequent business status analysis and decision-making.
[0170] To further improve the real-time performance and visualization of business status analysis, in some embodiments, step 106: the step of converting the business-independent feature matrix into a business feature analysis graph based on a streaming processing engine, and generating the decision strategy corresponding to the business feature analysis graph in real time, includes:
[0171] Step 701: Create a corresponding virtual data channel for the business-independent feature matrix in the streaming engine.
[0172] In step 701, a virtual data channel refers to an independent processing path allocated to a specific data stream in the streaming engine, ensuring the isolation and orderliness of data processing.
[0173] In this embodiment, a dedicated processing channel is created for the input matrix data in the streaming engine, and corresponding computing resources and cache space are configured to provide a stable data flow environment for subsequent analysis.
[0174] Step 702: In the virtual data channel, the business-independent feature matrix is decomposed into multiple data blocks, and each data block is mapped to an analysis node.
[0175] In step 702, a data block refers to a local data unit that divides the matrix by rows or columns. An analysis node is a graphical element that represents a single data block and contains all the feature information of that data block.
[0176] In this embodiment of the application, the feature matrix is divided into several data blocks in chronological order. Each data block contains indicator data with a fixed time span. Then, a corresponding analysis node is created for each data block. The node attributes include the request volume and processing time characteristics within that time period.
[0177] Step 703: Calculate the association strength between the analysis nodes, and establish the connection relationship between the analysis nodes based on the association strength.
[0178] In step 703, association strength refers to the degree of similarity in business characteristics between nodes. Connection relationship refers to the visual link between nodes established based on association strength.
[0179] In this embodiment, the similarity between every two analysis nodes is calculated in terms of both request volume and processing time. Nodes with similarity exceeding a set threshold are connected, and the thickness of the connection line reflects the strength of the association.
[0180] Step 704: Based on the connection relationship, construct a business feature analysis graph.
[0181] In this embodiment, all analysis nodes are arranged in chronological order, and connection lines are drawn based on the established connection relationships to form a feature map that can reflect the changing patterns of business status.
[0182] Step 705: Generate a corresponding decision strategy based on the topological structure characteristics of the business feature analysis graph.
[0183] In step 705, the topological structure features of the business feature analysis graph refer to the network structure characteristics formed by the connection relationships between the analysis nodes. These characteristics originate from the node connection relationships established based on the association strength, and specifically include quantitative indicators such as the connection density between nodes, the distribution of connection weights, and network clustering features.
[0184] In this embodiment of the application, special patterns such as densely connected areas and isolated nodes in the feature map are analyzed. When an abnormal connection pattern is detected, corresponding control instructions such as server expansion and traffic restriction are automatically generated.
[0185] Here is a specific example:
[0186] During a flash sale on e-commerce platform K, the system inputs a pre-constructed business-specific feature matrix into a streaming engine for real-time analysis. This matrix contains adjusted request counts and payment completion times over a 15-minute period, with one row per minute for a total of 15 rows. The streaming engine first creates a dedicated processing channel for the matrix data to ensure that the data processing is not interfered with by other tasks. The matrix is divided into three data blocks spanning 5 minutes, and each data block is converted into an analysis node. The node attributes include the average number of requests and payment completion time within that 5-minute period. The first node corresponds to the data from the first 5 minutes, with an average of 1420 requests per minute and an average payment completion time of 235 milliseconds; the second node corresponds to the data from the middle 5 minutes, with an average of 1560 requests per minute and an average payment completion time of 265 milliseconds; and the third node corresponds to the data from the last 5 minutes, with an average of 1680 requests per minute and an average payment completion time of 310 milliseconds. The similarity between nodes is calculated using the formula: Similarity = 1 - [|Request Count Difference| / (Maximum Request Count Difference) + |Payment Time Difference| / (Maximum Payment Time Difference)] / 2. Here, the request count difference and payment time difference refer to the differences in corresponding attributes between the two nodes. The maximum request count difference is taken as the difference between the maximum and minimum request counts in the matrix, which is 260 times / minute. The maximum payment time difference is taken as 90 milliseconds. The calculated similarity between node 1 and node 2 is 0.72, between node 2 and node 3 is 0.65, and between node 1 and node 3 is 0.58. A similarity threshold of 0.6 is set, and connections are established between nodes 1-2 and 2-3, with the thickness of the connection line proportional to the similarity. The constructed business feature analysis graph shows that the payment completion time of node 3 has significantly increased and the connection strength has weakened. After detecting this abnormal pattern, the system automatically generates decision strategies of "immediately increase server resources by 20%" and "start the queuing system," which are directly issued to the resource management platform for execution via API calls.
[0187] In this embodiment, the method achieves real-time graphical display of business status through a streaming processing engine; the feature map based on node association can intuitively reveal the evolution trend of system operation status; and the automated strategy generation mechanism ensures rapid response to abnormal situations, effectively improving the stability and reliability of the business system.
[0188] Figure 2 A schematic diagram of the structure of a big data-based business analysis system provided in this application embodiment is shown below. Figure 2 As shown, the system includes:
[0189] The acquisition module 21 is used to acquire user click stream data, payment transaction data, and device electromagnetic interference signals.
[0190] The first generation module 22 is used to generate multi-source heterogeneous business time-series data based on the user click stream data and the payment transaction data.
[0191] The conversion module 23 is used to convert the electromagnetic interference signal of the equipment into the frequency domain characteristics of the equipment vibration, and generate a collaborative data stream based on the frequency domain characteristics of the equipment vibration and the multi-source heterogeneous service time series data. The collaborative data stream includes mixed service characteristics.
[0192] The decoupling module 24 is used to decouple the hybrid service characteristics using the independent component analysis algorithm to obtain traffic density index and bandwidth index.
[0193] The construction module 25 is used to dynamically correlate the traffic density index and the bandwidth index to construct a service-independent feature matrix.
[0194] The second generation module 26 is used to convert the business-independent feature matrix into a business feature analysis graph based on a streaming processing engine, and to generate the decision strategy corresponding to the business feature analysis graph in real time.
[0195] Figure 2 The aforementioned big data-based business analysis system can perform... Figure 1 The implementation principle and technical effects of the big data-based business analysis method described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the big data-based business analysis system in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.
[0196] In one possible design, Figure 2 The big data-based business analysis system shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;
[0197] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.
[0198] The processing component 32 is used to perform the above. Figure 1 The embodiment describes a business analysis method based on big data.
[0199] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-described method.
[0200] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0201] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.
[0202] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.
[0203] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.
[0204] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.
[0205] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a business analysis method based on big data.
[0206] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0207] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A business analysis method based on big data, characterized in that, include: Acquire user clickstream data, payment transaction data, and device electromagnetic interference signals; Based on the user clickstream data and the payment transaction data, multi-source heterogeneous business time-series data is generated. The electromagnetic interference signal of the equipment is converted into the frequency domain characteristics of equipment vibration. Based on the frequency domain characteristics of equipment vibration and the multi-source heterogeneous service time series data, a collaborative data stream is generated. The collaborative data stream includes hybrid service characteristics. The characteristics of the hybrid services are decoupled using the independent component analysis algorithm to obtain traffic density and bandwidth indicators; The traffic density index and the bandwidth index are dynamically correlated to construct a service-independent feature matrix; Based on the streaming processing engine, the business-independent feature matrix is converted into a business feature analysis graph, and the decision strategy corresponding to the business feature analysis graph is generated in real time. The method of decoupling the hybrid service characteristics using independent component analysis (ICA) to obtain traffic density and bandwidth metrics includes: The separation factor is input into a pre-trained blind source separation model, and the traffic density signal source and bandwidth signal source are separated from the mixed service characteristics through the blind source separation model; Extract the first fluctuation trajectory corresponding to the number of requests per minute from the traffic density signal source, analyze the first fluctuation trajectory, and obtain the traffic density index; Extract the second fluctuation trajectory corresponding to the payment completion time from the bandwidth signal source, analyze the second fluctuation trajectory, and obtain the bandwidth index.
2. The method according to claim 1, characterized in that, The step of inputting the separation factor into a pre-trained blind source separation model, and separating the traffic density signal source and bandwidth signal source from the mixed service features through the blind source separation model, includes: The flow density weight and bandwidth weight in the separation factor are used as separation criteria and set in the decoupling layer of the blind source separation model; The decoupling layer is used to orthogonally project the hybrid service features to generate a feature vector space, which includes traffic density feature components and bandwidth feature components. The separation hyperplane is determined based on the distribution difference between the flow density characteristic component and the bandwidth characteristic component; Using the separating hyperplane, the feature vector space is divided into a first feature subspace and a second feature subspace; Extract the flow density signal source from the first feature subspace and extract the bandwidth signal source from the second feature subspace.
3. The method according to claim 2, characterized in that, The step of using the separating hyperplane to divide the feature vector space into a first feature subspace and a second feature subspace includes: Calculate the projection distance from each data point in the feature vector space to the separating hyperplane along the normal vector direction of the separating hyperplane; The boundary region is determined based on the sign of the projected distance; Based on the boundary region, the feature vector space is divided into two orthogonal subspaces; The orthogonal subspace containing the clustered region of the flow density feature components in the two orthogonal subspaces is defined as the first feature subspace; The orthogonal subspace containing the region where the bandwidth feature components are clustered is defined as the second feature subspace.
4. The method according to claim 1, characterized in that, The generation of a collaborative data stream based on the equipment vibration frequency domain characteristics and the multi-source heterogeneous service time series data includes: The vibration frequency domain characteristics of the equipment are superimposed with the multi-source heterogeneous service time-series data according to the sampling time points to generate superimposed waveform data. Through a signal injection mechanism, the superimposed waveform data is re-encapsulated into a service data packet according to the transmission protocol of the service data stream; Align the physical layer feature timestamps in the service data packets with the service data timestamps using execution timestamps; Based on the aligned business data packets, a collaborative data stream containing mixed business characteristics is output.
5. The method according to claim 1, characterized in that, The dynamic correlation between the traffic density index and the bandwidth index to construct a service-independent feature matrix includes: Establish a sliding time window, and within the sliding time window, calculate the fluctuation coupling degree between the flow density index and the bandwidth index; Based on the aforementioned fluctuation coupling degree, the flow density index value and bandwidth index value are weighted respectively; The weighted traffic density and bandwidth values are arranged in time series to generate a business-independent feature matrix.
6. The method according to claim 1, characterized in that, The process, based on a streaming processing engine, converts the business-independent feature matrix into a business feature analysis graph and generates decision strategies corresponding to the business feature analysis graph in real time, including: Create corresponding virtual data channels for the business-independent feature matrix in the streaming processing engine; In the virtual data channel, the business-independent feature matrix is decomposed into multiple data blocks, and each data block is mapped to an analysis node; Calculate the association strength between the analysis nodes, and establish the connection relationship between the analysis nodes based on the association strength; Based on the aforementioned connection relationships, a business feature analysis graph is constructed; Based on the topological characteristics of the business feature analysis graph, a corresponding decision-making strategy is generated.
7. A business analysis system based on big data, characterized in that, include: The acquisition module is used to acquire user clickstream data, payment transaction data, and device electromagnetic interference signals. The first generation module is used to generate multi-source heterogeneous business time-series data based on the user clickstream data and the payment transaction data; The conversion module is used to convert the electromagnetic interference signal of the equipment into the frequency domain characteristics of the equipment vibration, and generate a collaborative data stream based on the frequency domain characteristics of the equipment vibration and the multi-source heterogeneous service time series data. The collaborative data stream includes mixed service characteristics. The decoupling module is used to decouple the hybrid service characteristics using an independent component analysis algorithm to obtain traffic density and bandwidth indicators. A construction module is used to dynamically correlate the traffic density index and the bandwidth index to construct a service-independent feature matrix; The second generation module is used to convert the business-independent feature matrix into a business feature analysis graph based on the streaming processing engine, and generate the decision strategy corresponding to the business feature analysis graph in real time. The method of decoupling the hybrid service characteristics using independent component analysis (ICA) to obtain traffic density and bandwidth metrics includes: The separation factor is input into a pre-trained blind source separation model, and the traffic density signal source and bandwidth signal source are separated from the mixed service characteristics through the blind source separation model; Extract the first fluctuation trajectory corresponding to the number of requests per minute from the traffic density signal source, analyze the first fluctuation trajectory, and obtain the traffic density index; Extract the second fluctuation trajectory corresponding to the payment completion time from the bandwidth signal source, analyze the second fluctuation trajectory, and obtain the bandwidth index.
8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a big data-based business analysis method as described in any one of claims 1 to 6.
9. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a big data-based business analysis method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Service big data stream processing system and method based on feature analysis and medium
CN117390069A