Data analysis system and analysis method

Abstract

The invention discloses a data analysis system and a data analysis method. JavaScript for collecting user behaviors is introduced into a website to process browsed pages, click events and download files of a user, and the browsed pages, the click events and the download files are reported to a server side after passing through a specified interface; and binding the user unique identifiers of the software and the browser, and reporting the bound user unique identifiers to a server side through a specified interface. And the server side obtains and analyzes the binding relationship and the user behavior events of the browser and the software, so that the user behavior data can be collected in real time in multiple channels, the latest user dynamic state is obtained in time, and the timeliness and correlation of the data are ensured. The ClickHouse database is used for data storage, high-performance query capability is achieved, large-scale data can be processed, and rapid analysis is supported. The data processing module can clean original data, remove noise and abnormal values, improve data quality and perform business logic processing to ensure reliability and accuracy of analysis results. And providing an interface for calling and displaying a front-end interface.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis, and specifically relates to a data analysis system and an analysis method. Background Art

[0002] As enterprises go deeper into digital transformation, the collection and analysis of various user behavior data becomes increasingly important. In today's rapidly developing business environment, enterprises must accurately grasp the needs and preferences of users in order to remain invincible in the fierce market competition. The effective use of user behavior data can not only help enterprises gain a deeper understanding of consumer behavior patterns and decision-making processes, but also provide strong support for enterprises in product development and service optimization.

[0003] By analyzing user behavior data, companies can identify user pain points and needs and convert this information into actual product improvements and service enhancements. For example, companies can adjust product functions, add features required by users, or optimize user experience based on analysis results to improve customer satisfaction. In addition, companies can also develop more precise marketing strategies based on data analysis to ensure that information is pushed to target users through appropriate channels at the right time, thereby improving conversion rates.

[0004] In the prior art, for example, the technical solution described in the patent with publication number CN112418688A is: through the enterprise data management method based on the enterprise service portal, the various business data and basic information resources of the enterprise are integrated to help the enterprise service department to provide services to the enterprise quickly and efficiently. The specific implementation includes collecting enterprise data from the data source, evaluating the collected enterprise data, storing the evaluated enterprise data in the enterprise data management database, then processing the stored enterprise data, and finally building an enterprise data management statistical analysis module.

[0005] However, existing data analysis tools often seem powerless when faced with increasingly complex user data. For example, companies usually need to collect user behavior data from multiple channels such as the company's official website, software systems, third-party payment systems, content publishing platforms, etc., and clean, integrate and analyze the data after collection to obtain valuable business insights. Therefore, in the existing technology, there are analysis tools that often only support the integration of a single or limited number of data sources, which makes it difficult to cope with the increasingly diverse data sources of enterprises. When processing large-scale data, it is often impossible to achieve efficient analysis of real-time data due to the limitations of the technical architecture. Summary of the invention

[0006] The purpose of the present invention is to provide a data analysis system and analysis method to solve the problem in the prior art mentioned in the background technology that the analysis tools are difficult to cope with the increasingly diverse data sources of enterprises and cannot achieve efficient analysis of real-time data when processing large-scale data.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A data analysis system includes a data collection module: user behavior data is collected from multiple channels in real time through a standardized interface; a data storage module: a ClickHouse database is used to store and manage large-scale data for fast data reading and analysis; a data processing module: a back-end PHP and Laravel framework is used in combination with a front-end React framework to clean, organize and perform multi-dimensional analysis on the collected user data; a result display module: the front-end user interface displays various data analysis results in real time by calling the back-end API interface, and users can view the analysis results through visual reports and graphical interfaces.

[0009] According to the above technical solution, the data collection module includes a routing classification module and a data source parameter configuration module; wherein the routing classification module automatically selects a suitable path for data access according to the pre-configured routing rules of various data sources, thereby ensuring that the data collection process is fast and efficient; the data source parameter configuration module performs parameter analysis and configuration on data from different sources, determines the exact source of the data, and selects an appropriate processing strategy based on the source information.

[0010] A data analysis method, the analysis method comprising the following steps:

[0011] Step S1, data collection: collecting user behavior data from multiple channels in real time through a data collection module;

[0012] Step S2, data storage: the collected data will first be transmitted to the server and stored in the ClickHouse database of the data storage module;

[0013] Step S3, data processing; the data processing module cleans and preliminarily analyzes the stored data; the cleaned data will be further processed by business logic;

[0014] Step S4, result display: the processed analysis results will be provided to the front end through the API interface, and the front end user interface can dynamically display various analysis results according to their needs.

[0015] According to the above technical solution, in step S1, the data collection channel includes collecting user behavior data in real time from the company's official website, software system, third-party payment platform, and content publishing platform through the API interface. When users use these platforms, their behavior data will be automatically captured and sent to the data storage module.

[0016] According to the above technical solution, the data formats collected by the data collection module include JSON, XML and CSV.

[0017] According to the above technical solution, the user behavior data of the website and the user behavior data of the software are uploaded in a specified format through the API interface provided by the server; the system on the server adopts the PHP programming language for processing, firstly corrects the data collected in real time (data type conversion, date conversion, data structure conversion, etc.) and discards the data that does not meet the configuration requirements, and after the preliminary processing is completed, it is temporarily placed in the redis memory database as a memory cache; the memory cache is set by setting a specified time (such as 1 a.m.) or a data volume threshold (such as a cumulative total of 5,000). When the threshold is reached, the system will trigger a batch write operation, take the cached data out of the memory, and combine it into a data format for batch insert data, and finally write it to the data storage module.

[0018] According to the above technical solution, in step S2, when the data in the cache is written into the data storage module, in ClickHouse, the data will be stored in columnar storage and unnecessary I / O operations will be reduced; the data will be compressed when written to ClickHouse to reduce storage space occupancy.

[0019] Furthermore, in order to speed up data query, Clickhouses is an open source column storage database that speeds up query by using primary key indexes based on user behavior data, such as time, user unique identifier, event name, etc.

[0020] According to the above technical solution, in step S3, data processing includes performing a multi-dimensional analysis of events based on user behavior events, and the multi-dimensional analysis includes event data analysis and transaction habit and preference analysis; wherein the event data analysis includes time dimension analysis, user behavior preference analysis, and function usage frequency; the transaction habit and preference analysis includes transaction amount analysis, payment method preference, and user life cycle analysis.

[0021] According to the above technical solution, time dimension analysis is used to analyze the time distribution of events and find out the peak hours or trend changes; user behavior preference analysis analyzes the user's operation behavior to obtain their function usage preferences; function usage frequency: statistical analysis of the usage frequency of different functions helps companies understand user behavior habits;

[0022] Transaction amount analysis is used to count users' transaction amounts and analyze consumption habits of different user groups; payment method preference analysis is used to analyze the payment methods used by users in different scenarios, providing a basis for companies to optimize payment processes; user life cycle analysis tracks indicators such as users' first purchases, repurchase rates, and churn rates to analyze users' life cycle value and help companies formulate customer operation strategies.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] In the present invention, by collecting user behavior data in real time through multiple channels, the latest user dynamics can be obtained in time to ensure the timeliness and relevance of the data. ClickHouse database is used for data storage, which has high-performance query capabilities, can process large-scale data, and support fast analysis. The data processing module can clean the raw data, remove noise and outliers, improve data quality, and perform business logic processing to ensure the reliability and accuracy of the analysis results. The final analysis results can provide support for business decisions, help companies identify trends, optimize strategies, improve user satisfaction and increase operating income. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The data processing flow chart of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Embodiment 1

[0028] like Figure 1 As shown, a data analysis method comprises the following steps:

[0029] Step S1, data collection: collecting user behavior data from multiple channels in real time through a data collection module;

[0030] Step S2, data storage: the collected data will first be transmitted to the server and stored in the ClickHouse database of the data storage module;

[0031] Step S3, data processing; the data processing module cleans and preliminarily analyzes the stored data; the cleaned data will be further processed by business logic;

[0032] Step S4, result display: the processed analysis results will be provided to the front end through the API interface, and the front end user interface can dynamically display various analysis results according to their needs.

[0033] In the present invention, by collecting user behavior data in real time through multiple channels, the latest user dynamics can be obtained in time to ensure the timeliness and relevance of the data. ClickHouse database is used for data storage, which has high-performance query capabilities, can process large-scale data, and support fast analysis. The data processing module can clean the raw data, remove noise and outliers, improve data quality, and perform business logic processing to ensure the reliability and accuracy of the analysis results. The final analysis results can provide support for business decisions, help companies identify trends, optimize strategies, improve user satisfaction and increase operating income.

[0034] Embodiment 2

[0035] This embodiment is a further refinement of the first embodiment.

[0036] A data analysis system, including a data collection module: collecting user behavior data in real time from multiple channels such as a company's official website, software system, third-party payment platform, content publishing platform, etc. through a standardized interface;

[0037] Data storage module: Based on an efficient storage solution, the ClickHouse database is used to store and manage large-scale data to support fast data reading and analysis.

[0038] Data processing module: Through the back-end PHP and Laravel framework, combined with the front-end React framework, the collected user data is cleaned, sorted and multi-dimensionally analyzed. The multi-dimensional analysis process includes: the PHP program reads the user behavior event data collected from each end from the ClickHouse database, such as executing SQL:

[0039] select count(distinct dev_id)from analytics.web_soft_events

[0040] where fired_at between'2024-06-01 00:00:00'and'2024-06-3023:59:59';

[0041] This statement will get the total number of all users in June. According to this query method, by adding some specific conditions, we can get the number of website visits, button clicks, downloads, purchases, refunds, etc. in the time dimension; we can also use a single user as the dimension; for example, execute SQL:

[0042] select*from aomei_analytics.am_web_soft_events

[0043] wherefired_at between'2024-06-01 00:00:00'and'2024-06-3023:59:59'

[0044] and dev_id='dev_id_a34bd6dc-7243-44c5-b3c0-99bd7c50ff57'orderbyfired_at

[0045] By getting the activity records of a certain user on all the terminals of the company, we can see the entire activity records of the user. In this way, we can get data of various dimensions, for example, by first obtaining the unique identifiers of the number of first-time visitors to a certain page URL, and then querying the number of unique identifiers of these people in the download event, and then querying the unique identifiers of the software installed in the software event through the unique identifiers of the user bound to the browser and software reported by the software, we can get the funnel data from access, download, and installation of the software. Similarly, we can also filter the distribution of people on each terminal, the sales brought by each web page, software function, etc., and deduce the sales, access, download and other data brought by the company's related authors in combination with the company's article (web page) publishing system. We can also make natural calculations based on the directly queried data, such as dividing the purchase volume of a website by the visit volume to get the ratio of visits to purchases, etc. PHP, as the client of ClickHouse, obtains the results, calculates and organizes them into a reasonable data structure, and returns the relevant data to the front end for display in the form of an API. In this way, multi-dimensional analysis of user behavior data can be achieved.

[0046] Result display module: The front-end user interface displays various data analysis results in real time by calling the back-end API interface. Users can view the analysis results through visual reports and graphical interfaces.

[0047] System architecture: It adopts a front-end and back-end separated architecture design. The front-end uses the React framework to develop the user interface, and the back-end uses Laravel to provide API interfaces, supporting flexible modular development and system expansion.

[0048] The data collection module is the foundation of the entire system, ensuring that data from multiple sources can be seamlessly integrated into the analysis system. This module collects user behavior data in real time from channels such as the company's official website, software system, third-party payment platform, and content publishing platform through standardized interfaces. In order to cope with the diversity and complexity of different data sources, the data collection module has designed a flexible adapter system that can automatically select adaptation strategies based on the characteristics of different data sources, thereby ensuring the speed and accuracy of data collection.

[0049] The adapter system includes:

[0050] Routing classification module: Routing classification of reported data according to different data sources (such as browsers, software, third-party payment platforms, content publishing platforms, etc.). This module automatically determines the source of data by analyzing the metadata in the data packet, such as HTTP request header, API path, request method, etc., and routes it to the corresponding adapter processor.

[0051] The system pre-configures routing rules for various data sources. For example, data from a browser may be reported through an HTTP POST request, while data from a software system may be reported through a specific API path. The routing classification module automatically selects the appropriate path for data access based on these rules, ensuring that the data collection process is fast and efficient.

[0052] Data source parameter configuration module:

[0053] Parse and configure parameters of data from different sources, determine the exact source of the data, and select appropriate processing strategies based on the source information. This module can automatically identify and parse key parameters in data packets, such as device number, device type, timestamp, unique parameters of system configuration, etc., to determine which specific application or subsystem the data comes from.

[0054] By analyzing the parameters in the data packet in detail, the system can accurately identify the source of the data. For example, when processing data from a third-party payment platform, the system will determine the specific data source based on the transaction ID and payment method, and call the corresponding adapter module for processing. In this way, the adapter system can flexibly respond to data from different sources, which not only ensures the accuracy of data collection, but also improves the overall processing efficiency of the system.

[0055] Workflow of the adapter system: Data routing:

[0056] When data is reported from different channels, the system first uses the routing classification module to preliminarily identify and classify the data. The system routes the data to the corresponding adapter module according to the preset routing rules (such as API path, request method, etc.).

[0057] Parameter parsing: After the route classification is completed, the data enters the data source parameter configuration module. Here, the system will parse the data packet in detail and extract key parameters. These parameters will be used to further determine the specific source of the data and select the appropriate processing strategy.

[0058] Adapter selection: Based on the information provided by the data source parameter configuration module, the system will automatically select the corresponding adapter. These adapters are special processing modules for different data sources, which can standardize, clean and convert data to ensure that the data can be seamlessly integrated into the subsequent analysis process.

[0059] Data processing and storage: The data processed by the adapter will be transmitted to the data storage module (such as ClickHouse database). The system will further clean and analyze the data to ultimately achieve comprehensive integration and utilization of the data.

[0060] Advantages and innovations of the adapter system: The design of the adapter system greatly improves the flexibility and scalability of the data collection module of the present invention. Through automatic routing classification and data source parameter configuration, the system can cope with complex and diverse enterprise data sources and achieve efficient data collection and processing. In addition, the modular design of the system allows enterprises to flexibly expand according to their own needs, add new data sources or modify existing adapters, thereby maintaining the efficiency and accuracy of the system.

[0061] Embodiment 3

[0062] This embodiment is a further refinement of the second embodiment.

[0063] First, the system sets up a memory cache on the data collection end to temporarily store the data collected in real time, thereby reducing the pressure of direct access to the database. Then, the data will be written to the ClickHouse database in batches and stored in columns. The specific storage process is as follows:

[0064] Memory cache at the data collection end: In order to reduce the real-time pressure on the core database, the system sets up a memory cache at the initial stage of data collection, using redis as the memory database. When data is collected through the adapter system and prepared for storage, it will first be temporarily stored in the memory database redis. This mechanism helps buffer the frequency of data writing and avoids performance bottlenecks caused by direct writing to the database in high concurrency situations.

[0065] Implementation process:

[0066] 1. Data is collected from various data sources through API interfaces.

[0067] 2. The system first temporarily stores the data collected in real time into the memory database reidis.

[0068] 3. The memory cache sets a certain time or data volume threshold. When the threshold is reached, the system triggers a batch write operation to write the data in the cache to the ClickHouse database.

[0069] Implementation process:

[0070] 1. When enough data accumulates in the memory cache, the system triggers a batch write operation to transfer the data blocks to the ClickHouse database.

[0071] 2. In ClickHouse, data is stored in columnar format, which can improve reading speed and reduce unnecessary I / O operations when processing high-dimensional and massive data.

[0072] 3. Data is compressed when it is written. ClickHouse database software itself provides an efficient data compression algorithm, which can greatly reduce storage space usage. At the same time, compressed data also helps reduce network bandwidth usage and speed up data transmission.

[0073] The design of batch write operations further optimizes database performance and avoids the impact of frequent small data writes on the system.

[0074] Data reading and query

[0075] During the query process, the parallel processing capabilities and columnar storage structure of the ClickHouse database can quickly respond to complex query requests. The system will intelligently select appropriate data blocks for reading based on the query request, thereby avoiding full table scans and improving query efficiency.

[0076] Implementation process:

[0077] 1. When a user initiates a query request, the system first filters the data through the established index and selects the columns and rows that match the query conditions.

[0078] 2. Data is automatically decompressed when read and processed in parallel through a distributed computing cluster. ClickHouse's multi-threaded architecture ensures fast processing of large-scale data, and the system can maintain stable query performance even in high concurrency situations.

[0079] 3. Finally, the query results are returned to the user or the front-end display module through the API to achieve rapid data response.

[0080] Embodiment 4

[0081] In the data processing module, the specific processes and methods of data cleaning, processing and multi-dimensional analysis are as follows:

[0082] Data cleaning: Data cleaning is the first step to ensure data quality. Its purpose is to standardize and correct errors in raw data from different data sources. Specific methods include:

[0083] 1.1 Unified data format

[0084] Different data sources (such as browsers, software, third-party payment platforms, content publishing platforms) may use different data formats and structures. Through the data cleaning step, the system will format the data sent from each end to ensure that the data format is consistent. Common processing methods include:

[0085] Timestamp format conversion: unify the time format (such as unifying the time in different time zones into UTC format).

[0086] Type conversion of numerical data: ensure the consistency of numerical data types (such as converting floating-point numbers to integers or unifying numerical units and correcting necessary parameters, etc.).

[0087] 1.2 Data legitimacy verification

[0088] The system will verify and discard data that does not conform to the expected format, is missing key information, or is illegal:

[0089] Field integrity check: Checks whether the data contains all required fields, such as user ID, event type, timestamp, etc. If key fields are missing, the data will be marked as invalid and discarded.

[0090] Value range check: Verify whether the value of the data field is within a reasonable range. For example, whether the payment amount meets the standard and whether the timestamp is reasonable.

[0091] Data deduplication: Duplicate data is detected through unique identifiers (such as UUID or user ID) to prevent the same data from being recorded multiple times.

[0092] 1.3 Abnormal data processing

[0093] For abnormal or sudden error data, the system will take the following measures:

[0094] Outlier detection and processing: Identify abnormal data (such as parameter configuration mismatch mentioned above) through statistical methods and mark or discard them according to the set rules.

[0095] Repairing incorrect data: For some incorrect data, such as format mismatch or missing non-key fields, it can be automatically repaired through algorithms or default values.

[0096] 2. Data processing

[0097] The core of data processing is to further process and classify the cleaned data according to business logic for subsequent analysis. The specific steps include:

[0098] 2.1 User Behavior Event Processing

[0099] User behavior events reported by different applications and platforms will carry their own specific identifiers. By integrating the unique identifiers between different platforms (such as user ID, device ID, order ID, etc.), the user behavior data of each application is associated to generate a unified user behavior record:

[0100] Event classification: classify user operation behaviors (such as clicks, browsing, purchases, payments, etc.) and record them in standardized user behavior logs.

[0101] Behavior merging: Through unique identifiers, the behaviors of the same user on different applications, devices or platforms are merged to form a complete user path. This allows tracking the omnichannel behavior of the same user.

[0102] 2.2 Data Classification and Storage

[0103] According to business needs, the processed data is classified and stored:

[0104] Dimensional division: Classify and archive data according to multiple dimensions (such as time, device type, behavior type, etc.). Data of different dimensions can support different business needs, such as user behavior analysis, transaction habit analysis, etc.

[0105] Data aggregation: Aggregation operations are used to aggregate data of the same type, such as calculating the total transaction amount and user activity within a certain period of time.

[0106] 3. Multi-dimensional analysis

[0107] Multi-dimensional analysis is one of the core functions provided by the system, which supports complex business analysis based on user needs and generates visual analysis reports. Specific analysis methods include:

[0108] 3.1 Event Data Analysis

[0109] Based on user behavior events (such as login, browsing, purchase, payment, etc.), the system can perform multi-dimensional analysis on these events:

[0110] Time dimension analysis: Analyze the time distribution of events to find out the peak hours or trend changes. For example, the peak period of user access, the daily, weekly, and monthly distribution of transaction frequency, etc.

[0111] User behavior preference analysis: By analyzing the user's operation behavior, we can get their functional usage preferences. For example, we can analyze the functional modules that users use most often in the application, or the payment methods that they prefer during transactions.

[0112] Function usage frequency: Statistical analysis of the usage frequency of different functions helps companies understand user behavior habits and optimize product design.

[0113] 3.2 Analysis of trading habits and preferences

[0114] The system can analyze the user's transaction habits and preferences based on the user's transaction history data: Transaction amount analysis: Statistics on the user's transaction amount and analysis of the consumption habits of different user groups. Payment method preference: Analysis of the payment methods used by users in different scenarios (such as Alipay, WeChat Pay, etc.) to provide a basis for enterprises to optimize payment processes.

[0115] User life cycle analysis: By tracking indicators such as users’ first purchases, repurchase rates, and churn rates, we analyze the user’s life cycle value and help companies develop customer operation strategies.

[0116] 3.3 Customized multi-dimensional screening

[0117] Users can customize the filtering conditions according to their needs to conduct multi-dimensional combined analysis of data. For example: filter by time, event type, function module, transaction amount, etc. Segment user behavior by dimensions such as region, device type, operating system, etc.

[0118] 3.4 Data Visualization and Report Generation

[0119] The system supports the generation of various analysis reports and displays them in the form of charts, tables, etc.: Chart analysis: Generate line charts, bar charts, pie charts, etc. to intuitively display data trends and distribution. Custom reports: Generate customized analysis reports according to different needs, and support export functions to facilitate users to further process or report.

[0120] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0121] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data analysis system, characterized in that: It includes a data collection module: user behavior data is collected from multiple channels in real time through standardized interfaces; a data storage module: ClickHouse database is used to store and manage large-scale data for fast data reading and analysis; a data processing module: the collected user data is cleaned, sorted and analyzed in multiple dimensions through the back-end PHP and Laravel frameworks combined with the front-end React framework; a result display module: the front-end user interface displays various data analysis results in real time by calling the back-end API interface, and users can view the analysis results through visual reports and graphical interfaces.

2. A data analysis system and analysis method according to claim 1, characterized in that: The data collection module includes a routing classification module and a data source parameter configuration module; the routing classification module selects a path for data access according to the pre-configured routing rules of various data sources, thereby ensuring that the data collection process is fast and efficient; the data source parameter configuration module parses and configures parameters of data from different sources, and selects a processing strategy based on the source information.

3. A data analysis method, characterized in that: Using the system described in any one of claims 1 to 2 to perform data analysis, the analysis method comprises the following steps: Step S1, data collection: collecting user behavior data from multiple channels in real time through a data collection module; Step S2, data storage: the collected data will first be transmitted to the server and stored in the ClickHouse database of the data storage module; Step S3, data processing; the data processing module cleans and preliminarily analyzes the stored data; the cleaned data will be further processed by business logic; Step S4, result display: the processed analysis results will be provided to the front end through the API interface, and the front end user interface can dynamically display various analysis results according to their needs.

4. A data analysis method according to claim 3, characterized in that: In step S1, the data collection channel includes collecting user behavior data in real time from the company's official website, software system, third-party payment platform, and content publishing platform through the API interface. When users use these platforms, their behavior data will be automatically captured and sent to the data storage module.

5. A data analysis method according to claim 4, characterized in that: The data collection module collects data in formats including JSON, XML, and CSV.

6. A data analysis method according to claim 5, characterized in that: The website's user behavior data and the software's user behavior data are uploaded in a specified format through the API interface provided by the server; the server's system uses the PHP programming language for processing, first correcting the real-time collected data and discarding the data that does not meet the configuration requirements, and after the initial processing is completed, it is temporarily placed in the redis memory database as a memory cache; the memory cache sets a specified time or data volume threshold. When the threshold is reached, the system triggers a batch write operation, takes the cached data out of the memory, and combines it into a data format for batch insert data, and finally writes it to the data storage module.

7. A data analysis method according to claim 6, characterized in that: In step S2, when the data in the cache is written to the data storage module, the data will be stored in ClickHouse in columnar storage; the data is compressed when written to ClickHouse to reduce storage space occupancy.

8. A data analysis method according to claim 7, characterized in that: In step S3, data processing includes performing a multi-dimensional analysis of events based on user behavior events, and the multi-dimensional analysis includes event data analysis and transaction habit and preference analysis; wherein the event data analysis includes time dimension analysis, user behavior preference analysis, and function usage frequency; the transaction habit and preference analysis includes transaction amount analysis, payment method preference, and user life cycle analysis.

9. A data analysis method according to claim 8, characterized in that: Time dimension analysis is used to analyze the time distribution of events and identify peak periods or trend changes; user behavior preference analysis analyzes users' operating behaviors to derive their function usage preferences; function usage frequency: statistical analysis of the frequency of use of different functions helps companies understand user behavior habits.

10. A data analysis method according to claim 9, characterized in that: Transaction amount analysis is used to count users' transaction amounts and analyze consumption habits of different user groups; payment method preference analysis is used to analyze the payment methods used by users in different scenarios, providing a basis for companies to optimize payment processes; user life cycle analysis tracks users' first purchases, repurchase rates, and churn rates, analyzes users' life cycle value, and helps companies formulate customer operation strategies.

Images