Data processing method and device, electronic equipment, medium and product

Abstract

The invention discloses a data processing method and device, electronic equipment, a medium and a product, and the method comprises the steps: receiving a data analysis request which comprises a data query condition and analysis type indication information; and obtaining user behavior data matched with the data query condition, and performing integration processing on the user behavior data according to a user identifier corresponding to the user behavior data to obtain behavior sequence data of each user. Data analysis processing is conducted on the behavior sequence data of each user according to the analysis type indication information, a first analysis result is obtained, and the first analysis result comprises the data analysis result of each user. Through the method, the data analysis efficiency can be improved, and different data analysis requirements can be met.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a data processing method, a data processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] User Behavior Analysis Systems (BBAs) are popular analytics systems in the operations and data analytics industries. Their primary function is to modify or adjust business operational strategies based on the calculation results of various models. Key analytical models include funnel analysis, retention analysis, attribution analysis, path analysis, interval analysis, and Lifetime Value (LTV) analysis, among others.

[0003] Currently, a popular computing solution is to use ClickHouse (a columnar database management system specifically designed for high-performance data analysis and data warehousing applications) for user behavior analysis. In ClickHouse, user behavior data is distributed and stored across a cluster. However, as the data volume grows, query performance can no longer meet the demands for real-time queries. More and more analytical computations can only be performed offline, preventing the generation of operational metrics in real time. This results in a delay in making real-time operational strategies.

[0004] Therefore, improving the efficiency of data analysis has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a data processing method, apparatus, electronic device, medium, and product that can improve data analysis efficiency and meet different data analysis needs.

[0006] One embodiment of this application discloses a data processing method, the method comprising:

[0007] Receive a data analysis request, the data analysis request including data query conditions and analysis type indication information;

[0008] Obtain user behavior data that matches the data query conditions, and integrate the user behavior data according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of each user;

[0009] The behavioral sequence data of each user is analyzed and processed according to the analysis type indication information to obtain a first analysis result, which includes the data analysis result of each user.

[0010] One embodiment of this application discloses a data processing apparatus, which includes:

[0011] A transceiver unit is used to receive data analysis requests, which include data query conditions and analysis type indication information.

[0012] The acquisition unit is used to acquire user behavior data that matches the data query conditions, and to integrate and process the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user.

[0013] The processing unit is configured to perform data analysis processing on the behavioral sequence data of each user according to the analysis type indication information to obtain a first analysis result, wherein the first analysis result includes the data analysis result of each user.

[0014] One aspect of this application discloses an electronic device, which includes a processor adapted to implement one or more computer programs; and a computer-readable storage medium storing one or more computer programs adapted to be loaded by the processor and executed with respect to the data processing method described above.

[0015] One aspect of this application discloses a computer-readable storage medium storing one or more computer programs adapted to be loaded by a processor and executed by the above-described data processing method.

[0016] One embodiment of this application discloses a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the aforementioned data processing method.

[0017] In this embodiment, the task execution server receives a data analysis request, which includes data query conditions and analysis type indication information. User behavior data matching the data query conditions is acquired, and the user behavior data is integrated and processed according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data for each user. Since the task execution server's local storage space stores the user behavior data matching the data query conditions, it does not need to establish connections with other servers to acquire user behavior data, resulting in high data acquisition efficiency. Furthermore, integrating the user behavior data according to the user identifier allows all behavior data for a single user to be organized together for efficient analysis. Data analysis processing is performed on the behavior sequence data for each user according to the analysis type indication information to obtain a first analysis result, which includes the data analysis result for each user. Different data analysis modules can be invoked according to the analysis type indication information to achieve different data analyses. The data analysis types in this embodiment include, but are not limited to, funnel analysis, retention analysis, path analysis, attribution analysis, and interval analysis. In summary, the data processing method provided in this embodiment can not only meet different types of analysis needs but also effectively improve data analysis efficiency through user identifiers. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the network architecture of a data processing system disclosed in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a data processing method disclosed in an embodiment of this application;

[0021] Figure 3 This is a flowchart of data entry and query process for a StarRocks database disclosed in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the data processing flow of a target analysis module disclosed in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of a multi-function thread processing user behavior sequence data disclosed in an embodiment of this application;

[0024] Figure 6This is a schematic flowchart of a funnel analysis method disclosed in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the structure of a data processing device disclosed in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Before describing the embodiments of this application, the relevant terms involved in the data processing method will be explained:

[0029] User Behavior Analysis System: A user behavior analysis system is a popular analytical system in the operations and data analytics industries. Its main function is to modify or adjust enterprise operational strategies based on the calculation results of its various models. Key analytical models include funnel analysis, retention analysis, attribution analysis, path analysis, interval analysis, LTV analysis, and session analysis, among others.

[0030] Funnel analysis: Funnel analysis is a process-oriented data analysis method used to scientifically reflect user behavior and conversion rates at each stage from start to finish. This method is widely used in daily data operations and analysis work, such as traffic monitoring for website and app user behavior analysis, CRM systems, SEO optimization, product marketing, and sales. The most commonly used metrics in funnel analysis are conversion rate and churn rate, two complementary indicators. These two metrics can be used to assess user engagement and churn at different stages, thereby helping to optimize user experience and improve conversion rates.

[0031] In funnel analysis, commonly used metrics include conversion rate, median conversion time, and optimal conversion channel. Conversion rate refers to the percentage of users who proceed to the next step at each stage of the process. Funnel analysis is a process-oriented data analysis method used to scientifically reflect user behavior and conversion rates at each stage from start to finish. Median conversion time, especially in setting up conversion funnel models, measures the time required for each step of the conversion process when analyzing the user's purchase journey. This time metric helps analyze the overall conversion situation and the specific conversion rate and median conversion time at each step, thereby optimizing the user experience and purchase process. For example, in the complete user purchase process, from browsing products to completing payment, each step has an expected completion time; this time is the median conversion time. By analyzing this time, we can understand how long users spend at each step, thus optimizing the process and improving the conversion rate. Optimal conversion channel is one of the most important metrics in the analysis model, representing the fastest way to complete the conversion, such as traffic channels, payment methods, etc.

[0032] ClickHouse is a columnar database management system specifically designed for high-performance data analytics and data warehousing applications, designed to meet the needs of large-scale data analysis and reporting.

[0033] StarRocks is an open-source, next-generation, high-speed, full-scenario MPP (Massively Parallel Processing) database. It employs a new generation of elastic MPP architecture, efficiently supporting various data analysis scenarios, including multi-dimensional analysis, real-time analysis, and high-concurrency analysis with large datasets. StarRocks boasts outstanding performance, utilizing comprehensive vectorization technology, making it 3-5 times faster than similar products on average.

[0034] Currently, a popular computational implementation solution is to use ClickHouse for user behavior analysis. Funnel analysis is frequently used, primarily employing ClickHouse's `windowFunnel()` function for querying. However, ClickHouse suffers from poor performance in concurrency support and performs poorly in cluster environments. With increasing data volumes, ClickHouse's query performance can no longer meet the demands for real-time queries, forcing more and more analytical computations to be performed offline. Since operational metrics cannot be obtained in real time, there is a delay in making real-time operational strategies. Furthermore, ClickHouse's function library is insufficient. While ClickHouse boasts a rich function library that supports most analytical scenarios, the increasing complexity of operational analytics metrics means that ClickHouse's function library is gradually failing to meet the evolving needs of the industry. This makes it unsuitable for scenarios with complex and ever-changing query strategies.

[0035] For example, ClickHouse provides its own funnel analysis function, `windowFunnel`, which is defined as: within a defined sliding window, the function sequentially searches the event chain; the maximum number of events touched by the function on this event chain is the maximum funnel depth for the user. However, funnel analysis often requires finding the optimal conversion channel. Due to the limitations of ClickHouse's funnel function, this solution can only group the data before calculation, not group users based on the results during calculation. However, pre-calculation grouping is far from sufficient for operational analysis scenarios. Often, users have multiple channel values, and we need to find the channel that prioritizes conversion. If the data is grouped before calculation, it cannot meet the requirement of finding the highest priority conversion channel among multiple groups. Funnel analysis mainly focuses on conversion rate. Taking channels as an example, conversion rate is tied to channels; that is, we need to frequently see which channel has the highest conversion rate. When a user has conversions through multiple channels, we need to select the optimal conversion channel. Current methods cannot directly provide the optimal forwarding channel, resulting in low data analysis efficiency.

[0036] Therefore, based on the above problems, in order to perform data analysis efficiently, this application proposes a data processing method. This method is executed by an electronic device, which may include a terminal or a server. The terminal may include, but is not limited to, smartphones, tablets, smart wearable devices, smart voice interaction devices, smart home appliances, personal computers, vehicle terminals, smart cameras, virtual reality devices (such as AR (Augmented Reality) devices), etc. The server may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0037] In this embodiment, the electronic device executing the data processing method is any task execution server in the task execution server cluster of the target database. The target database can be a StarRocks database. Specifically, the method includes: ① Receiving a data analysis request, which includes data query conditions and analysis type indication information. ② Obtaining user behavior data matching the data query conditions, and integrating the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user. Since the local storage space stores user behavior data matching the data query conditions, it is not necessary to establish a connection with other servers to obtain user behavior data, resulting in high data acquisition efficiency. In addition, integrating the user behavior data according to the user identifier can organize all the behavior data of a user together for efficient analysis. ③ Performing data analysis processing on the behavior sequence data of each user according to the analysis type indication information to obtain a first analysis result, which includes the data analysis result of each user. Different data analysis modules can be called according to the analysis type indication information to achieve different data analyses. The data analysis types in this embodiment include, but are not limited to, funnel analysis, retention analysis, path analysis, attribution analysis, interval analysis, lifetime value analysis, and session analysis. In summary, the data processing method provided in this application can not only meet different types of analysis needs, but also effectively improve data analysis efficiency through user identification.

[0038] Please see Figure 1This is a schematic diagram of the network architecture of a data processing system disclosed in an embodiment of this application. The data processing system may include multiple first electronic devices 101, second electronic devices 102, and terminal devices 103. The first electronic devices 101 and the terminal devices 103 are connected via a network, and the first electronic devices 101 and the second electronic devices 102 are connected via a network, such as through a local area network, a wide area network, or the mobile Internet.

[0039] The first electronic device 101 can be any task execution server in the target database's task execution server cluster, and the second electronic device 102 can be the target database's task management server. The first electronic device 101 is primarily responsible for executing data processing methods, including: receiving a data analysis request from the second electronic device 102 containing data query conditions; querying user behavior data matching the data query conditions; integrating and processing the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user among at least one user; and performing data analysis and processing on the behavior sequence data for each user according to analysis type indication information to obtain a first analysis result including the data analysis result for each user. Alternatively, any first electronic device 101 from multiple first electronic devices 101 can be selected as a result integration server, which integrates the first analysis results from each first electronic device 101 to obtain the target analysis result.

[0040] The second electronic device 102 is mainly responsible for receiving the data analysis request generated by the terminal device 103 and sending the data analysis request to the first electronic device 101; receiving the target analysis result returned by the first electronic device 101 as a result integration server, and returning the target analysis result to the terminal device 103 so that the user of the terminal device 103 can know the analysis result of this data analysis request.

[0041] The first electronic device 101 and the second electronic device 102 in this application embodiment can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This application embodiment does not limit them in this way.

[0042] In this embodiment, terminal device 103 is the terminal device corresponding to the user who requests data analysis. The user configures relevant parameters for data analysis, including data query conditions, through the display interface of terminal device 103. Different types of data analysis requests require different data query conditions. After the user completes the configuration on the display interface, the backend of terminal device 103 generates a data analysis request based on the configured parameters and sends the data request to the second electronic device 102 for data processing. Terminal device 103 in this application embodiment includes, but is not limited to, smartphones, tablets, smart wearable devices, smart voice interaction devices, smart home appliances, personal computers, in-vehicle terminals, smart cameras, virtual reality devices (such as AR (Augmented Reality) devices), etc., and this application does not impose any limitations on these devices.

[0043] Optionally, Figure 1 This is merely an illustrative representation of the network architecture of a data processing system and is not intended to be limiting. For example, Figure 1 The first electronic device 101 or the second electronic device 102 can be deployed as a node in the blockchain network, or the first electronic device 101 or the second electronic device 102 can be connected to the blockchain network, so that the first electronic device 101 or the second electronic device 102 can store the data involved in the data processing to the blockchain network to prevent the internal data from being tampered with, thereby ensuring data security.

[0044] In this embodiment of the application, in conjunction with a data processing system, the data processing method provided in this embodiment can generally include the following steps: Terminal device 103 sends a data analysis request to second electronic device 102; second electronic device 102 distributes the data analysis request to first electronic device 101 in a task execution server cluster belonging to the target database; first electronic device 101 receives the data analysis request, retrieves user behavior data matching the data query conditions included in the data analysis request from its local storage space, and integrates the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user among at least one user; performs data analysis processing on the behavior sequence data of each user according to the analysis type indication information to obtain a first analysis result including the data analysis result for each user. Finally, whenever a first electronic device 101 receives a data analysis request, it obtains a first analysis result, then uses any first electronic device 101 in the cluster as an integration server to integrate the various first analysis results to obtain a target analysis result, and returns the target analysis result to the second electronic device 102, which then returns the target analysis result to the terminal device 103. Since each first electronic device stores user behavior data matching the data query conditions in its local storage space, there is no need to establish a connection with the first electronic device to obtain user behavior data, thus achieving high data acquisition efficiency. Furthermore, by integrating user behavior data based on user identifiers, all behavior data of a single user can be organized together for efficient analysis. Simultaneously, different data analysis modules can be invoked based on analysis type indication information, enabling various data analyses. In summary, the data processing method provided in this application embodiment not only meets different types of analysis needs but also effectively improves data analysis efficiency through user identifiers.

[0045] Optionally, the data processing system may further include a third electronic device 104. The third electronic device 104 is used to store the module file of the custom data analysis module provided in this application embodiment. When the first electronic device 101 calls the target data analysis module corresponding to the analysis type indication information to perform data analysis processing on the behavioral sequence data of each user, it obtains the module file of the target data analysis module from the third electronic device 104 through the file address of the target data analysis module, and then runs the target data analysis module according to the module file of the target data analysis module. The third electronic device 104 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This application embodiment does not limit this.

[0046] In one possible implementation, when the data processing method is applied to the StarRocks database, StarRocks consists of two types of nodes: FE and BE. FE, short for FrontEnd, is the front-end node of StarRocks, responsible for managing metadata, managing client connections, and performing query planning and scheduling. BE, short for BackEnd, is the back-end node of StarRocks, responsible for data storage, computation execution, and replica management. Correspondingly, the first electronic device in the data processing architecture is the BE, the second electronic device is the FE, and the terminal device 103 is the client. The corresponding data processing flow is as follows: The client initiates a data analysis request to StarRocks. The StarRocks FE receives the query request and generates a distribution execution plan. The distribution execution plan distributes the data analysis request to each BE. After receiving the data analysis request, the BE begins data processing and obtains the corresponding first analysis result. Then, it selects any BE in StarRocks as the result integration server, receives data analysis result integration requests from other BEs, performs result integration processing, and obtains the target analysis result. The target analysis result is sent to the FE, which returns the target analysis result to the client for display or further analysis.

[0047] It should be noted that the data collection and processing described in this application should be strictly in accordance with the requirements of relevant laws and regulations, obtaining the informed consent or separate consent of the personal information subject, and conducting subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0048] The data management method proposed in the embodiments of this application will be described in detail below.

[0049] Please see Figure 2 This is a flowchart illustrating a data processing method disclosed in an embodiment of this application. This data management method is applied to a task execution server cluster of a target database and can be executed by any task execution server in the cluster. The task execution server can be... Figure 1 For any of the first electronic devices 101 shown, the corresponding data processing method may specifically include, but is not limited to, the following steps:

[0050] S201: Receive a data analysis request. The data analysis request includes data query conditions and analysis type indication information.

[0051] In one possible implementation, the task execution server receives a data analysis request from the task management server of the target database. The target database could be a StarRocks database or a Doris database. Both types of databases allow for custom data analysis modules, thus satisfying diverse data analysis requests. After receiving the user's data analysis request, the task management server generates a distribution execution plan based on the request, determines which task execution servers in the task execution server cluster need to send the data analysis request to, and then executes the distribution execution plan to send the data analysis request to the corresponding task execution servers. In this embodiment, since the data storage task is distributed by the task management server during data storage, when determining the distribution execution plan for the data analysis request, the task management server can first determine which task execution servers store matching data based on the data analysis request, and then send the data analysis request to the corresponding task execution servers.

[0052] The data analysis request received by the task execution server includes data query conditions, which may include one or more of the following: time information (mainly time range, specifically start time and time window), event information (mainly event identifier), and specific fields. These specific fields correspond to different content in different data analysis requests. For example, in funnel analysis, specific fields may include grouping fields, association attribute fields, external filter fields, etc. Additionally, the data analysis request also carries analysis type indication information, which primarily indicates the type of data analysis request. In this embodiment, the types of data analysis requests include, but are not limited to, funnel analysis, retention analysis, attribution analysis, path analysis, interval analysis, LTV analysis, etc. Given the target database provided in this application embodiment, developers can customize more data analysis types to meet their data analysis needs.

[0053] S202: Obtain user behavior data that matches the data query conditions, and integrate and process the user behavior data according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of each user.

[0054] In one possible implementation, the task execution server retrieves user behavior data matching the data query conditions from local storage. Before retrieving the user behavior data matching the data query conditions from local storage, the data storage process is described as follows: The task management server receives a data storage request sent by the client. The data storage request includes a first user identifier and the user behavior data to be stored. The task management server hashes the first user identifier to obtain a target hash. Then, it obtains the identifiers of each task execution server in the task execution server cluster, compares each task execution server's identifier with the target hash, and obtains the task execution server corresponding to the identifier that matches the target hash. The task management server sends the data storage request to the task execution server. The task execution server responds to the data storage request and associates the first user identifier and the user behavior data to be stored in local storage. This ensures that all data for the same user is stored in the same task execution server, avoiding the performance latency caused by the need for different task execution servers to exchange user data during data querying or data analysis due to the dispersed storage of the same user's data.

[0055] In addition, when storing data, the task execution server stores the user attribute information corresponding to the user behavior data to be stored in the first data table, and stores the specific user behavior data in the user behavior data to be stored in the second data table. The first and second data tables are linked and stored together by user identifiers. In this way, when a data analysis request is received, the task execution server can filter the data based on the two data tables and the user attribute information carried in the data analysis request. Since the two data tables are stored locally on the same task execution server, the query efficiency can be greatly improved.

[0056] Based on the aforementioned storage process, after receiving a data analysis request, the task execution server retrieves user behavior data matching the data query conditions from its local storage space. The task execution server directly retrieves the corresponding user behavior data from the second data table based on the event identifier in the data query conditions. Since a task execution server may store user behavior data for multiple users, after obtaining a large amount of user behavior data, it can integrate the user behavior data based on user identifiers to obtain at least one user's behavior sequence data. The integration process includes aggregating user behavior data based on user identifiers, that is, aggregating user behavior data with the same user identifier together, and then sorting each behavior record according to the occurrence time of each behavior record in the user behavior data to obtain at least one user's behavior sequence data. Each behavior record in the obtained behavior sequence data includes the behavior steps, the behavior occurrence timestamp, etc. Specifically, a user's behavior sequence data includes two parts: a user identifier and a user behavior sequence. The user behavior sequence includes multiple behavior records, each connected by a delimiter. Assuming a task execution server has behavior sequence data for 3 users, the behavior sequence data for the 3 users can be found in Table 1:

[0057] Table 1

[0058] User ID User Behavior Sequence User ID 1 Behavior record 1 / Behavior record 2 / ... / Behavior record x User ID 2 Behavior record 1 / Behavior record 2 / ... / Behavior record y User ID 3 Behavior record 1 / Behavior record 2 / ... / Behavior record z

[0059] As shown in Table 1, if the task execution server stores user behavior data for multiple users, the resulting behavior sequence data for each user is displayed in list form, including two columns: one for user identifier and one for user behavior sequence. Each row represents the user behavior data for one user, and each behavior record in each row is arranged in chronological order from earliest to latest. The number of behavior records can vary for different users.

[0060] Optionally, if the data query conditions also include user attribute filtering information, the first data table can be queried using the user identifier corresponding to each user to determine whether the user attribute information and user attribute filtering information stored for that user are consistent. If they are consistent, the user behavior data is retained; if they are inconsistent, the user behavior data can be deleted.

[0061] The target database provided in this application embodiment has excellent cluster concurrency capabilities and features partitioning and bucketing mechanisms. Partitioning involves storing data separately using a partition key; both data insertion and querying operate only on the data within the corresponding partition, resulting in higher efficiency. For example, data can be partitioned by day. Bucketing, on the other hand, distributes data evenly across the servers in the cluster using a hash algorithm during data insertion. This evenly distributes the insertion load across all servers and also distributes the data retrieval load evenly across each server during queries.

[0062] The reasons for choosing the user identifier as the key field when storing or integrating data are as follows: 1. The user identifier is the most distinctive field. This allows for even distribution of data across servers in the cluster during data entry, avoiding data skew and distributing query pressure evenly across all servers. Furthermore, during analysis and computation, CPU resources are fully utilized, and computational pressure is evenly distributed across all servers in the cluster. 2. Considering the business implementation approach, since all analysis modules are user-centric, user data needs to be aggregated before formal analysis. Because storage is based on the user identifier, all data for a single user must reside on the same machine. Using the user identifier as the key allows data aggregation on the local disk, avoiding network communication aggregation between servers and eliminating the additional time costs associated with network interactions, resulting in higher efficiency.

[0063] Please see Figure 3 This is a flowchart illustrating the data import and query process of the StarRocks database. Figure 3 (a) in the text represents the data storage process. Specifically, the storage client sends a data storage request to the FE, the FE performs a hash calculation on the user identifier in the data storage request to determine which BE the data should be stored on, and sends the data to be stored to the corresponding BE. Figure 3 (b) in the diagram represents the data analysis process. Specifically, the analysis client sends a data analysis request to the FE, and the FE generates a corresponding distribution execution plan based on the data analysis request. The pressure of the data analysis request is then evenly distributed to each BE according to the distribution execution plan. Since the stored procedure is based on user identifier storage, the data analysis request will not be concentrated in one BE when generating the distribution execution plan. Therefore, the data analysis pressure can be evenly distributed, improving the efficiency of data analysis.

[0064] S203: Perform data analysis on the behavioral sequence data of each user according to the analysis type indication information to obtain the first analysis result, which includes the data analysis result of each user.

[0065] In one possible implementation, after the task execution server determines the behavior sequence data of each user, it obtains the file address of the target data analysis module corresponding to the analysis type indication information. Using this file address, it retrieves the module file of the target data analysis module from the file server and runs the target data analysis module according to the module file. After the target data analysis module runs successfully, the task execution server directly calls the target data analysis module to perform data analysis processing on the behavior sequence data of each user, obtaining the data analysis results for each user. In other words, the target data analysis module inputs the behavior sequence data of each user, analyzes and processes it, obtains the data analysis results for each user, and integrates the data analysis results of each user to obtain the first analysis result.

[0066] In one possible implementation, the target data analysis module is created based on a function implementation template and registered to the file address based on a registration function. The target data analysis module includes any one of the following: funnel analysis module, retention analysis module, path analysis module, attribution analysis module, interval analysis module, lifetime value analysis module, and session analysis module. The functions corresponding to the target data analysis module can be customized. The target database used in this embodiment supports custom functions, specifically UDAF functions (User-defined aggregate functions, a general term for custom functions developed based on business scenarios, designed to extend function capabilities), which are fully customizable. The development and registration process for UDAF functions is as follows: 1. First, create a JAR project and import the function implementation template. The template is a Java file containing create, update, serialize, merge, and finalize methods, each responsible for different processing steps; 2. Package the project into a JAR file; 3. Upload the JAR file to a file server that can communicate with the StarRocks server for storage and obtain the file address. 4. Register functions using the `createAGGREGATE FUNCTION` statement. In this statement, `file` is configured as the HTTP file address of the JAR file, and `symbol` is the path to the function within the JAR file. Through the development and registration of these functions, user behavior analysis modules such as funnel analysis, retention analysis, path analysis, attribution analysis, and interval analysis are implemented. The implementation methods of each module are basically the same, differing only in logic; the code processing architecture is also consistent. Compared to the ClickHouse database, the functions provided in ClickHouse can no longer directly retrieve the required operational analysis results. More and more analytical models require writing complex and difficult-to-maintain SQL queries. Some analytical model metrics even require multiple queries to obtain results, leading to increasingly complex query SQL, extremely poor readability, and slow query speeds. This is not only time-consuming and laborious to write but also incurs significant costs in later SQL maintenance. Using the functions defined in this application, fast data retrieval can be achieved.

[0067] Please see Figure 4This is a flowchart illustrating the data processing of a target analysis module disclosed in this application, including the following steps: Step 1: Use the `split` function to segment the behavior sequence data of each user, dividing the behavior sequence data from a string into a processable event array using a delimiter, and sorting them according to timestamps. Step 2: The function thread corresponding to the target analysis module begins parsing and calculating the event array. Step 3: After all the behavior sequence data of all users in the task execution server has been processed, they are integrated to obtain the first analysis result corresponding to the task execution server. Each time the function thread completes the calculation of a user's behavior sequence data, it temporarily stores the corresponding data analysis result in memory.

[0068] In this embodiment, the number of function threads for each task execution server can be customized to allow multiple function threads to process the behavior sequence data of multiple users in parallel. Specifically, this can be set through the `pipeline_dop` parameter, using the SQL statement: `set global variables pipeline_dop = number of threads`. Figure 5 The diagram illustrates how multiple function threads process user behavior sequence data, with one function thread processing the behavior sequence data of one user. Figure 5 As shown, after each function thread completes its calculations, a dataset (i.e., the first analysis result) can be obtained by integrating the results. This dataset includes the data analysis results for each user. Further, the task execution server needs to send a merge request to the result integration server (i.e., the task execution server cluster designated as the result integration server). The first analysis results from each task execution server in the task execution server cluster are merged and organized into the result integration server. The result integration server processes the integrated target analysis results, converting them into data formats such as JSON and XML, and returns the results to the party that submitted the data analysis request.

[0069] In one possible implementation, after each task execution server obtains its first execution result, the process of sending these results to the result integration server is as follows: A data analysis result integration request is sent to the result integration server, the request including the first analysis result; the result integration server, based on the data analysis result integration request, integrates the first analysis results sent by each task execution server to obtain the target analysis result corresponding to the data analysis request. Here, the result integration server can be any task execution server in the task execution server cluster. The result integration server can be user-specified or determined based on the number of function threads currently executing within the task execution server.

[0070] In this embodiment, the process is described as follows: In the target database, the task execution server receives a data analysis request sent by the task management server of the target database. The data analysis request carries data query conditions and analysis type indication information. User behavior data matching the data query conditions is retrieved from local storage space, and the user behavior data is integrated and processed according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of at least one user. Since the local storage space stores the user behavior data matching the data query conditions, it is not necessary to establish a connection with other servers to retrieve user behavior data, resulting in high data retrieval efficiency. In addition, integrating the user behavior data according to the user identifier can organize all the behavior data of a user together for efficient analysis. The target data analysis module corresponding to the analysis type indication information is called to perform data analysis processing on the behavior sequence data of each user to obtain a first analysis result, which includes the data analysis result of each user.

[0071] Furthermore, different data analysis modules can be invoked based on the analysis type indication information to achieve different data analyses. The data analysis types in this application embodiment include, but are not limited to, bucket analysis, retention analysis, path analysis, attribution analysis, and interval analysis. In summary, the data processing method provided in this application embodiment can not only meet different types of analysis needs but also effectively improve data analysis efficiency through user identification.

[0072] The above embodiments illustrate that various types of data analysis requests can be processed in the target database. To further demonstrate the advantages of this processing method, we will use funnel analysis as an example to illustrate the data processing method of this application embodiment. Please refer to [link to relevant documentation]. Figure 6 This is a flowchart illustrating a funnel analysis method disclosed in an embodiment of this application. The funnel analysis method is applied to a StarRocks database, which includes a task management server and a task execution server. Correspondingly, the funnel analysis method may include, but is not limited to, the following steps:

[0073] S601: Receives a funnel analysis request sent by the task management server. The funnel analysis request includes data query conditions and funnel analysis instruction information.

[0074] In one possible implementation, the user configures data query conditions for funnel analysis on the terminal device's display interface. The terminal device's backend then generates a funnel analysis request based on the user's configuration. This request carries the corresponding data query conditions, which may include the action steps, query time, and relevant special fields. These special fields can be grouping fields, related attribute fields, external filter fields, etc. The corresponding action steps differ depending on the funnel model. Common funnel models include recruitment funnels, marketing funnels, user growth funnels, information dissemination funnels, and sales funnels. Additionally, the funnel analysis request also carries funnel analysis indication information to indicate that the request is for performing funnel analysis.

[0075] Furthermore, the backend of the terminal device sends the funnel analysis request to the database's task management server. Based on the funnel analysis request, the task management server generates a corresponding distribution execution plan and distributes the funnel analysis request to the corresponding task execution server.

[0076] S602: Obtain user behavior data matching the data query conditions from the local storage space, and integrate the user behavior data according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of each user in at least one user; the behavior sequence data of each user includes N behavior records arranged in chronological order, where N is an integer greater than or equal to 1.

[0077] In one possible implementation, after receiving a funnel analysis request, the task execution server retrieves user behavior data matching the data query conditions from local storage. This user behavior data is associated with the behavioral steps of the funnel analysis. After identifying the user behavior data, the task execution server clusters the user behavior data based on the user identifiers corresponding to the user behavior data, obtaining at least one user's behavior data set. Then, it sorts and integrates the behaviors in each user's behavior data set in chronological order, obtaining the behavior sequence data for each user in at least one user. Each user's behavior sequence data includes N behavior records arranged in chronological order, where N is an integer greater than or equal to 1. Each behavior record appears as an array, and an array includes at least the behavior step, the execution time of the behavior step, and the grouping value of the behavior step. The grouping value of the behavior step is carried in the funnel analysis request, and the grouping value can be the operating system, traffic channel, payment method, etc.

[0078] S603: Perform funnel analysis on the N behavior records of each user's behavior sequence data in order from early to late according to the timestamp, to obtain the funnel set of each user, where the funnel set includes M funnels.

[0079] In this context, the first behavior record in each of the M funnels corresponds to the initial behavior step of the funnel analysis, where M is the number of behavior records in the N behavior records that are located before the first behavior record and correspond to the initial behavior step; the first behavior record corresponds to the terminating behavior step of the funnel analysis, or the first behavior record is the last behavior record in the N behavior records; and one of the funnels includes the first behavior record.

[0080] In one possible implementation, when the analysis type indication information includes funnel analysis indication information, the task execution server calls the funnel analysis module to split the behavior sequence data of each user, obtaining N behavior records for each user. Assume the behavior steps of a funnel are: Startup -> Homepage -> Details -> Download. See Table 2, which shows multiple behavior records for a user:

[0081] Table 2

[0082] Serial Number Behavioral steps Grouping values 1 start up Operating System 1 2 start up Operating System 2 3 front page Operating System 1 4 Details Operating System 2 5 front page Operating System 2 6 Details Operating System 1 7 download Operating System 1 8 download Operating System 2

[0083] In Table 2, the three columns represent the serial number (the size of the serial number indicates the time), the action step, and the group value, respectively.

[0084] Based on the N behavioral records obtained for each user, the funnel analysis module corresponding to the funnel analysis indication information is invoked. The specific process of performing funnel analysis on the N behavioral records for each user in order of timestamp from earliest to latest is as follows:

[0085] 1. Query each user's N behavior records in chronological order (earliest to latest) to find the first behavior record corresponding to the initial behavior step of the funnel analysis, and generate the first funnel for each user. Add the first behavior record to the first funnel. If the initial behavior step of the funnel analysis is "Start," find the first behavior record corresponding to "Start" from the N behavior records, generate a funnel, and add this behavior record to the funnel. Since the funnel only contains the initial behavior step of the funnel analysis at this time, the conversion depth is 1.

[0086] 2. Iterate through the N behavior records in chronological order from earliest to latest, starting with the first behavior record. If the behavior step corresponding to the currently iterated second behavior record is different from the initial behavior step, add the second behavior record to the first funnel. Since the funnel now only contains the initial behavior step and the second behavior record from the funnel analysis, the conversion depth is incremented by 1. If the behavior step corresponding to the second behavior record is not the terminating behavior step of the funnel analysis, continue iterating through the behavior records after the second behavior record until the behavior step corresponding to the currently iterated third behavior record is the terminating behavior step. Add the third behavior record to the first funnel and stop iterating. At the same time, add the first funnel for each user to each user's funnel set. If the behavior step corresponding to the second behavior record is the terminating behavior step of the funnel analysis, stop iterating and add the first funnel for each user to each user's funnel set. As more behavior steps are added to the funnel, the conversion depth increases. If the behavior step of a behavior record is the same as the terminating behavior step of the funnel analysis, the maximum conversion depth is reached.

[0087] 3. If the behavior step corresponding to the second behavior record in the current iteration is the same as the initial behavior step, then a second funnel is generated for each user, and the second behavior record is added to the second funnel. This can be understood as follows: during the iteration process, whenever a behavior record is encountered whose behavior step matches the initial behavior step in the funnel analysis, a new funnel is generated.

[0088] 4. Continue iterating through the N behavior records in ascending order of timestamps, starting with the second behavior record. If the behavior step corresponding to the currently iterated fourth behavior record is different from the initial behavior step, add the fourth behavior record to the second funnel. If the behavior step corresponding to the fourth behavior record is not the terminating behavior step, continue iterating until the behavior step corresponding to the currently iterated fifth behavior record is the terminating behavior step. Add the fifth behavior record to the second funnel and stop iterating. Add the second funnel for each user to each user's funnel set. If the behavior step corresponding to the fourth behavior record is the terminating behavior step of the funnel analysis, stop iterating and add the second funnel for each user to each user's funnel set.

[0089] Based on the behavior records in Table 2, the traversal process is illustrated with an example. Starting from row 1, the behavior record in row 1 corresponds to the step "Start," which is the same as the initial behavior step in the funnel analysis. Therefore, a first funnel is generated, and "Start" is added to the first funnel, resulting in a conversion depth of 1. Continuing to traverse row 2, the behavior record in row 2 corresponds to the step "Start," which is the same as the initial behavior step in the funnel analysis. Therefore, a second funnel is generated, and "Start" is added to the second funnel, still resulting in a conversion depth of 1. Continuing to traverse row 3, the behavior record in row 3 corresponds to the step "Home," which is different from the initial behavior step in the funnel analysis. Therefore, "Home" is added to both the first and second funnels, resulting in a conversion depth of 2. Continuing to traverse row 4, the behavior record in row 4 corresponds to the step "Details," which is different from the initial behavior step in the funnel analysis. Therefore, "Details" is added to both the first and second funnels, resulting in a conversion depth of 3. Continuing to iterate through line 5, the action record in line 5 corresponds to the step "Home". Although this is different from the initial action step in the funnel analysis, it is skipped because no adjacent "Start" can be found for "Home". Continuing to iterate through line 6, the action record in line 6 corresponds to the step "Details". This is different from the initial action step in the funnel analysis, so "Details" is added to both the first and second funnels to cover the "Details" in line 4. Since no new steps are added, the conversion depth remains unchanged at 3. Continuing to iterate through line 7, the action record in line 7 corresponds to the step "Download". This is different from the initial action step in the funnel analysis, and "Download" is the terminating action step in the funnel analysis, so the iteration stops. "Download" is added to the first funnel, and the conversion depth changes to 4. This results in two funnels, with the first funnel reaching a conversion depth of 4.

[0090] It's important to note that during the process of traversing behavior records to determine the funnel, a new funnel is generated each time an initial behavior step occurs. Consider that the computational complexity increases exponentially with the number of funnels. Within funnels at the same step level, the funnel step with the largest timestamp is most likely to reach subsequent funnel steps. Therefore, given the synchronization limit, funnel steps with smaller timestamps can be overwritten to compress the funnel steps. As shown in Table 2, rows 4 and 6 both contain "details," so the "details" in row 6 can be used to overwrite the "details" in row 4. In a practical implementation, funnel steps with smaller timestamps can be marked. This allows marked funnels to be skipped when determining the optimal transformation sequence, significantly reducing traversal complexity.

[0091] Based on the above explanation, the steps can be summarized as follows: After determining the behavior records of each user, traverse them in chronological order. For each behavior record encountered, update the funnel set for each user. This process includes determining whether an update is needed, which steps in the funnel to update, and after the update, determining whether the maximum conversion depth has been achieved. If the maximum conversion depth is achieved, stop traversing. The resulting data analysis includes the maximum conversion depth, the optimal conversion sequence, and the corresponding grouping value sequence. Conversely, if no conversion depth is achieved, traverse all behavior records and take the grouping value and behavior sequence corresponding to the larger conversion depth as the data analysis result.

[0092] S604: Determine the first analysis result based on the behavioral records included in each of the M funnels for each user.

[0093] In one possible implementation, after identifying M funnels during the traversal process, the optimal conversion behavior sequence for each user is determined based on the behavior records in the M funnels included in each user's funnel set, arranged in ascending order of timestamps. Then, based on the grouping values ​​corresponding to the behavior records in each user's optimal conversion behavior sequence, a grouping value sequence for each user is determined. Each behavior record has one grouping value, and the optimal conversion behavior sequence includes multiple behavior records. The grouping values ​​of these multiple behavior records are combined to obtain each user's grouping value sequence. Finally, each user's optimal conversion behavior sequence, grouping value sequence, and conversion depth are used as the data analysis results for each user. The conversion depth is determined based on the number of behavioral steps included in the optimal conversion behavior sequence. The first analysis result is determined based on each user's data analysis results. The specific data in the data analysis results can be determined according to the data analysis requirements and can be one or more of the optimal conversion behavior sequence, grouping value sequence, and conversion depth.

[0094] Furthermore, the optimal conversion behavior sequence for each user is determined based on the behavior records in the M funnels included in each user's funnel set, arranged from latest to earliest timestamps. Specifically, this involves sorting the behavior records in the M funnels of each user's funnel set from earliest to latest timestamps, resulting in L behavior records for each user. The last behavior record among the L behavior records is the behavior record corresponding to the termination behavior step, which occurs when the termination behavior step of the funnel analysis is encountered during traversal. Alternatively, the last behavior record among the L behavior records may be the last behavior record among the N behavior records, meaning that none of the user's behavior records contain the termination behavior step of the funnel analysis, indicating that all N behavior records have been traversed. Here, L is an integer greater than or equal to 0, and L is less than or equal to N. The corresponding I behavior records within the update time window for each user's L behavior records are then obtained, where I is an integer greater than or equal to 0, and I is less than or equal to L. The process involves querying I behavior records in ascending order of timestamps and based on the behavioral steps included in the funnel analysis. The goal is to identify the target behavior sequence that matches these steps and use it as the optimal conversion behavior sequence for each user. During the traversal, if the funnel analysis reaches its termination step, the target behavior sequence is formed by reversing the order of the behavior records corresponding to that termination step, sequentially finding the target behavior records that match the funnel analysis steps, and then using the corresponding behavior steps to create the optimal conversion behavior sequence for each user. If the funnel analysis does not reach its termination step, the target behavior sequence is formed by selecting the first few steps from the I behavior records that correspond to the funnel analysis steps. For example, if the funnel's behavior steps are: Startup -> Homepage -> Details -> Download, and there is no termination step "Download," then the query would be Startup -> Homepage -> Details, and so on, ensuring each user finds a target behavior sequence. The update time window mentioned above is based on the specific settings of the funnel analysis. For instance, only conversions completed within the update time window are valid; conversions outside the update time window are considered invalid.

[0095] As shown in Table 2, when traversing to the 7th row, the traversal stops. At this point, it is necessary to find the optimal conversion sequence in reverse order. The optimal conversion sequence is: {[7、Download、Operating System 1],[6、Details、Operating System 1],[3、Home、Operating System 1],[2、Startup、Operating System 2]}; the grouping value sequence is {1,1,1,2}; the conversion depth is 4.

[0096] In one possible implementation, the data analysis request also includes filtering instructions. In funnel analysis, the filtering instructions can refer to external filtering instructions, which is a filtering method after obtaining the data analysis results for each user. Therefore, the grouping value sequence of each user can also be filtered according to the grouping value indicated by the filtering instructions, filtering out S users from at least one user, where S is an integer greater than or equal to 0; the data analysis results of the S users are used as the first analysis result. For example, if the grouping value indicated by the filtering instructions is [1,1,1], and the task execution server obtains the data analysis results of 3 users, and the grouping value sequences of the 3 users are [1,2,1], [1,1,1], and [1,1,1], filtering the 3 users by the grouping value [1,1,1], then the user corresponding to [1,2,1] is deleted, and the data analysis results of the latter 2 users are obtained, and the data analysis results of the latter 2 users are used as the first analysis result. Compared with data analysis using the ClickHouse database, the data processing method provided in this application embodiment can support external filtering (i.e., the above process), which can improve data processing efficiency. Specifically, external filtering involves filtering the grouped values ​​from the funnel to check if they meet set conditions, discarding those that don't. In other words, external filtering uses the grouped value sequence of the optimal conversion sequence to determine if it meets the conditions; those that don't are not included in the result merging. For example, in one application scenario, if we need to see if the number of conversions from an advertising exposure channel is superior to other channels, and if 1000 users completed conversions, external filtering can be used to obtain the number of users from the advertising exposure channel that achieved the best conversion among these 1000 users.

[0097] In one possible implementation, the data analysis request also includes conversion time interval indication information. The method further includes: obtaining the timestamps of each behavioral step in each user's optimal conversion behavior sequence; determining the conversion time interval for each user based on the timestamps of each behavioral step; for example, if user A's optimal conversion behavior sequence includes three steps, calculating the first time interval before the first and second steps, calculating the second time interval before the second and third steps, and adding the first and second time intervals together to obtain user A's conversion time interval. The shortest conversion time interval among all users is taken as the optimal conversion time interval. Alternatively, each user's conversion time interval can also be a sequence, representing the conversion interval between each behavioral step. The optimal conversion time interval can be understood as the best time interval between a node and the next node in the user conversion path, which maximizes user conversion efficiency and user experience. For example, in the shopping process of an e-commerce platform, the best time interval from the product details page to adding to the cart, and the best time interval from adding to the cart to submitting the order, are all important factors affecting whether users ultimately complete the purchase. By analyzing these time intervals, the shopping process can be optimized, and the user's purchase conversion rate can be improved. The ClickHouse database cannot directly output the conversion time interval, but the data processing method provided in this application embodiment can directly output the conversion time interval, thus improving data processing efficiency.

[0098] This application embodiment specifically describes the funnel analysis process. Based on the above description, it can be seen that the target database used in this application embodiment supports the implementation of custom functions, solving the increasingly complex operational indicator calculation needs in the field of user behavior analysis and catering to the development trend of operational technology. For example, some commonly occurring indicators in funnel analysis, including optimal conversion sequence, external screening, correlation attributes, conversion time interval calculation, etc., can all be quickly statistically calculated using this method.

[0099] Based on the above method embodiments, this application also provides a schematic diagram of the structure of a data processing apparatus. See also... Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 7 The data processing device 700 shown can operate the following units:

[0100] The transceiver unit 701 is used to receive data analysis requests, the data analysis requests including data query conditions and analysis type indication information;

[0101] The acquisition unit 702 is used to acquire user behavior data that matches the data query conditions, and to integrate and process the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user.

[0102] The processing unit 703 is configured to perform data analysis processing on the behavioral sequence data of each user according to the analysis type indication information to obtain a first analysis result, wherein the first analysis result includes the data analysis result of each user.

[0103] In one possible implementation, when the processing unit 703 performs data analysis processing on the behavioral sequence data of each user according to the analysis type indication information and obtains the first analysis result, it specifically performs the following steps:

[0104] Obtain the file address of the target data analysis module corresponding to the analysis type indication information;

[0105] The module file of the target data analysis module is obtained from the file server using the file address;

[0106] The target data analysis module is run according to its module file, and the behavioral sequence data of each user is analyzed and processed by the target data analysis module to obtain the first analysis result.

[0107] In one possible implementation, the target data analysis module is created based on a function implementation template and registered to the file address based on a registration function. The target data analysis module includes any one of the following: funnel analysis module, retention analysis module, path analysis module, attribution analysis module, interval analysis module, lifetime value analysis module, and session analysis module.

[0108] In one possible implementation, the transceiver unit 701 is further configured to receive a data storage request, which is sent based on a matching result between a target hash and the identifiers of multiple task execution servers. The data storage request includes a first user identifier and user behavior data to be stored, and the target hash is generated based on the first user identifier.

[0109] The processing unit 703 is also configured to respond to the data storage request by associating and storing the first user identifier and the user behavior data to be stored in the local storage space.

[0110] In one possible implementation, the behavior sequence data of each user includes N behavior records arranged in chronological order, where N is an integer greater than or equal to 1. When the processing unit 703 performs data analysis processing on the behavior sequence data of each user according to the analysis type indication information and obtains the first analysis result, it specifically performs the following steps:

[0111] When the analysis type indication information includes funnel analysis indication information, funnel analysis is performed on the N behavior records included in the behavior sequence data of each user in chronological order from earliest to latest timestamp to obtain the funnel set of each user; wherein, the funnel set includes M funnels, and the first behavior record included in each funnel corresponds to the initial behavior step of the funnel analysis, where M is the number of behavior records in the N behavior records that are located before the first behavior record and correspond to the initial behavior step; the first behavior record corresponds to the terminating behavior step of the funnel analysis, or the first behavior record is the last behavior record in the N behavior records; one of the funnels includes the first behavior record;

[0112] The first analysis result is determined based on the behavioral records included in each of the M funnels for each user.

[0113] In one possible implementation, when processing unit 703 performs funnel analysis on the N behavior records included in the behavior sequence data of each user in order from earliest to latest according to the timestamp, and obtains the funnel set of each user, it specifically performs the following steps:

[0114] Query the first behavior record corresponding to the initial behavior step of the funnel analysis among the N behavior records included in the behavior sequence data of each user in the order of timestamp from early to late, generate the first funnel for each user, and add the first behavior record to the first funnel;

[0115] The N behavior records are traversed in order of timestamp from earliest to latest, starting from the first behavior record.

[0116] If the action step corresponding to the second action record being traversed is different from the initial action step, then the second action record is added to the first funnel;

[0117] If the behavior step corresponding to the second behavior record is not the termination behavior step of the funnel analysis, then continue traversing until the behavior step corresponding to the currently traversed third behavior record is the termination behavior step, add the third behavior record to the first funnel, and stop traversing.

[0118] Add the first funnel of each user to the funnel set of each user.

[0119] In one possible implementation, the processing unit 703 is further configured to perform the following steps:

[0120] If the behavior step corresponding to the second behavior record currently being traversed is the same as the initial behavior step, then a second funnel is generated for each user, and the second behavior record is added to the second funnel;

[0121] The N behavior records are traversed in order of timestamp from earliest to latest, starting with the behavior record following the second behavior record.

[0122] If the action step corresponding to the fourth row record being traversed is different from the initial action step, then the fourth row record is added to the second funnel.

[0123] If the behavior step corresponding to the fourth behavior record is not the termination behavior step, then continue traversing until the behavior step corresponding to the currently traversed fifth behavior record is the termination behavior step, add the fifth behavior record to the second funnel, and stop traversing;

[0124] Add the second funnel of each user to the funnel set of each user.

[0125] In one possible implementation, when determining the first analysis result based on the behavior records included in each of the M funnels for each user, the determining unit 704 specifically performs the following steps:

[0126] Based on the behavior records in the M funnels included in each user's funnel set, and arranged in order from late to early timestamps, determine the optimal conversion behavior sequence for each user.

[0127] Based on the grouping values ​​corresponding to the behavior records included in the optimal conversion behavior sequence of each user, determine the grouping value sequence for each user;

[0128] The optimal conversion behavior sequence, grouping value sequence, and conversion depth for each user are used as the data analysis results for each user; wherein, the conversion depth is determined based on the number of behavioral steps included in the optimal conversion behavior sequence;

[0129] The first analysis result is determined based on the data analysis results of each user.

[0130] In one possible implementation, when determining the optimal conversion behavior sequence for each user based on the behavior records in the M funnels included in each user's funnel set, according to the timestamps from latest to earliest, the determining unit 704 specifically performs the following steps:

[0131] The behavior records included in the M funnels of each user's funnel set are sorted according to the timestamp from earliest to latest to obtain L behavior records for each user. The last behavior record in the L behavior records is the behavior record corresponding to the termination behavior step, or the last behavior record in the L behavior records is the last behavior record in the N behavior records; where L is an integer greater than or equal to 0, and L is less than or equal to N.

[0132] Obtain the corresponding I behavior records of each user's L behavior records within the update time window, where I is an integer greater than or equal to 0 and I is less than or equal to L;

[0133] Query the I behavior records according to the timestamp from late to early and the behavior steps included in the funnel analysis, and determine the target behavior sequence that matches the behavior steps included in the funnel analysis;

[0134] The target behavior sequence is taken as the optimal conversion behavior sequence for each user.

[0135] In one possible implementation, the data analysis request further includes filtering instruction information, and the processing unit 703 is further configured to perform the following steps:

[0136] Based on the grouping value indicated by the filtering instruction information, the data analysis results of each user include a sequence of grouping values ​​for filtering processing, and S users are filtered out from at least one user, where S is an integer greater than or equal to 0.

[0137] The data analysis results of the S users are used as the first analysis result.

[0138] In one possible implementation, the data analysis request further includes conversion time interval indication information, and the acquisition unit 702 is also used to acquire the timestamps of each behavior step in the optimal conversion behavior sequence for each user;

[0139] The determining unit 704 is further configured to determine the conversion time interval of each user based on the timestamps of each behavioral step included in each user; and to take the shortest conversion time interval of each user as the optimal conversion time interval.

[0140] It is understood that the functions of each functional unit of the data processing device provided in the embodiments of this application can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0141] In feasible embodiments, the data processing apparatus provided in this application can be implemented in software. The data processing apparatus can be stored in a memory and can be software in the form of programs and plug-ins. It includes a series of units, including a transceiver unit, an acquisition unit, a processing unit, and a determination unit. The transceiver unit, acquisition unit, processing unit, and determination unit are used to implement the data processing method provided in this application.

[0142] In other feasible embodiments, the data processing device provided in this application embodiment can also be implemented in a combination of hardware and software. As an example, the data processing device provided in this application embodiment can be a processor in the form of a hardware decoding processor, which is programmed to implement the data processing method provided in this application embodiment. For example, the processor in the form of a hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0143] This application embodiment, combining a transceiver unit, an acquisition unit, a processing unit, and a determination unit, includes the following method: the transceiver unit receives a data analysis request, the data analysis request carrying data query conditions and analysis type indication information; the acquisition unit acquires user behavior data matching the data query conditions, and integrates and processes the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user; the processing unit performs data analysis processing on the behavior sequence data for each user according to the analysis type indication information to obtain a first analysis result, the first analysis result including the data analysis result for each user. In this method, since the local storage space of the task execution server stores the user behavior data matching the data query conditions, it is not necessary to establish a connection with other task execution servers to acquire user behavior data, thus the data acquisition efficiency is high; in addition, integrating and processing the user behavior data according to the user identifier can organize all the behavior data of a user together for efficient analysis. At the same time, different data analysis modules can be called according to the analysis type indication information to achieve different data analyses. In summary, the data processing method provided by the application embodiment can not only meet different types of analysis needs, but also effectively improve data analysis efficiency through user identifiers.

[0144] Based on the descriptions of the method and device embodiments above, this application also provides an electronic device; this electronic device may be the first electronic device 101 mentioned above, specifically a task execution server. Please refer to... Figure 8 The electronic device may include at least a processor 801, an input interface 802, an output interface 803, and a computer-readable storage medium 804; wherein the processor 801, the input interface 802, the output interface 803, and the computer-readable storage medium 804 may be connected by a bus or other connection methods.

[0145] The computer-readable storage medium 804 can be stored in the memory of the electronic device. The computer-readable storage medium 804 is used to store computer programs, and the processor 801 is used to execute the computer programs stored in the computer-readable storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the electronic device. It is suitable for implementing one or more computer programs, specifically for loading and executing one or more computer programs to achieve corresponding method flows or corresponding functions.

[0146] This application also provides a computer-readable storage medium (Memory), which is a memory device in an electronic device used to store programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the electronic device and extended storage media supported by the electronic device. The computer-readable storage medium provides storage space that stores the operating system of the electronic device. Furthermore, the storage space also stores one or more computer programs (including program code) suitable for loading and execution by a processor. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0147] In this embodiment, the processor 801 can load and execute one or more computer programs stored in the computer-readable storage medium 804 to implement the corresponding steps of the aforementioned data processing method. Specifically, the computer program in the computer-readable storage medium 804 is loaded and executed by the processor 801 in the following steps:

[0148] Receive a data analysis request, the data analysis request including data query conditions and analysis type indication information;

[0149] Obtain user behavior data that matches the data query conditions, and integrate the user behavior data according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of each user;

[0150] The behavioral sequence data of each user is analyzed and processed according to the analysis type indication information to obtain a first analysis result, which includes the data analysis result of each user.

[0151] In one possible implementation, when the computer program in the computer-readable storage medium 804 is loaded and executed by the processor 801 to perform data analysis processing on the behavioral sequence data of each user according to the analysis type indication information and obtain a first analysis result, it is specifically used to perform the following steps:

[0152] Obtain the file address of the target data analysis module corresponding to the analysis type indication information;

[0153] The module file of the target data analysis module is obtained from the file server using the file address;

[0154] The target data analysis module is run according to its module file, and the behavioral sequence data of each user is analyzed and processed by the target data analysis module to obtain the first analysis result.

[0155] In one possible implementation, the target data analysis module is created based on a function implementation template and registered to the file address based on a registration function. The target data analysis module includes any one of the following: funnel analysis module, retention analysis module, path analysis module, attribution analysis module, interval analysis module, lifetime value analysis module, and session analysis module.

[0156] In one possible implementation, the computer program in the computer-readable storage medium 804 is further loaded by the processor 801 and executed in the following steps:

[0157] A data storage request is received, which is sent based on the matching result between a target hash and the identifiers of multiple task execution servers. The data storage request includes a first user identifier and user behavior data to be stored. The target hash is generated based on the first user identifier.

[0158] In response to the data storage request, the first user identifier and the user behavior data to be stored are associated and stored in the local storage space.

[0159] In one possible implementation, the behavior sequence data of each user includes N behavior records arranged in chronological order, where N is an integer greater than or equal to 1. The computer program in the computer-readable storage medium 804 is loaded and executed by the processor 801 to perform data analysis processing on the behavior sequence data of each user according to the analysis type indication information. When obtaining the first analysis result, it specifically performs the following steps:

[0160] When the analysis type indication information includes funnel analysis indication information, funnel analysis is performed on the N behavior records included in the behavior sequence data of each user in chronological order from earliest to latest timestamp to obtain the funnel set of each user; wherein, the funnel set includes M funnels, and the first behavior record included in each funnel corresponds to the initial behavior step of the funnel analysis, where M is the number of behavior records in the N behavior records that are located before the first behavior record and correspond to the initial behavior step; the first behavior record corresponds to the terminating behavior step of the funnel analysis, or the first behavior record is the last behavior record in the N behavior records; one of the funnels includes the first behavior record;

[0161] The first analysis result is determined based on the behavioral records included in each of the M funnels for each user.

[0162] In one possible implementation, when the computer program in the computer-readable storage medium 804 is loaded and executed by the processor 801 to perform funnel analysis processing on the N behavior records included in the behavior sequence data of each user in order from earliest to latest timestamps, and to obtain the funnel set of each user, it is specifically used to perform the following steps:

[0163] Query the first behavior record corresponding to the initial behavior step of the funnel analysis among the N behavior records included in the behavior sequence data of each user in the order of timestamp from early to late, generate the first funnel for each user, and add the first behavior record to the first funnel;

[0164] The N behavior records are traversed in order of timestamp from earliest to latest, starting from the first behavior record.

[0165] If the action step corresponding to the second action record being traversed is different from the initial action step, then the second action record is added to the first funnel;

[0166] If the behavior step corresponding to the second behavior record is not the termination behavior step of the funnel analysis, then continue traversing until the behavior step corresponding to the currently traversed third behavior record is the termination behavior step, add the third behavior record to the first funnel, and stop traversing.

[0167] Add the first funnel of each user to the funnel set of each user.

[0168] In one possible implementation, the computer program in the computer-readable storage medium 804 is further loaded by the processor 801 and executed in the following steps:

[0169] If the behavior step corresponding to the second behavior record currently being traversed is the same as the initial behavior step, then a second funnel is generated for each user, and the second behavior record is added to the second funnel;

[0170] The N behavior records are traversed in order of timestamp from earliest to latest, starting with the behavior record following the second behavior record.

[0171] If the action step corresponding to the fourth row record being traversed is different from the initial action step, then the fourth row record is added to the second funnel.

[0172] If the behavior step corresponding to the fourth behavior record is not the termination behavior step, then continue traversing until the behavior step corresponding to the currently traversed fifth behavior record is the termination behavior step, add the fifth behavior record to the second funnel, and stop traversing;

[0173] Add the second funnel of each user to the funnel set of each user.

[0174] In one possible implementation, the computer program in the computer-readable storage medium 804, when loaded and executed by the processor 801 to determine the first analysis result based on the behavioral records included in each of the M funnels for each user, specifically performs the following steps:

[0175] Based on the behavior records in the M funnels included in each user's funnel set, and arranged in order from late to early timestamps, determine the optimal conversion behavior sequence for each user.

[0176] Based on the grouping values ​​corresponding to the behavior records included in the optimal conversion behavior sequence of each user, determine the grouping value sequence for each user;

[0177] The optimal conversion behavior sequence, grouping value sequence, and conversion depth for each user are used as the data analysis results for each user; wherein, the conversion depth is determined based on the number of behavioral steps included in the optimal conversion behavior sequence;

[0178] The first analysis result is determined based on the data analysis results of each user.

[0179] In one possible implementation, when the computer program in the computer-readable storage medium 804 is loaded and executed by the processor 801 to determine the optimal conversion behavior sequence for each user based on the behavior records in the M funnels included in the funnel set for each user, in order from latest to earliest timestamps, it specifically performs the following steps:

[0180] The behavior records included in the M funnels of each user's funnel set are sorted according to the timestamp from earliest to latest to obtain L behavior records for each user. The last behavior record in the L behavior records is the behavior record corresponding to the termination behavior step, or the last behavior record in the L behavior records is the last behavior record in the N behavior records; where L is an integer greater than or equal to 0, and L is less than or equal to N.

[0181] Obtain the corresponding I behavior records of each user's L behavior records within the update time window, where I is an integer greater than or equal to 0 and I is less than or equal to L;

[0182] Query the I behavior records according to the timestamp from late to early and the behavior steps included in the funnel analysis, and determine the target behavior sequence that matches the behavior steps included in the funnel analysis;

[0183] The target behavior sequence is taken as the optimal conversion behavior sequence for each user.

[0184] In one possible implementation, the data analysis request further includes filtering instruction information, and the computer program in the computer-readable storage medium 804 is further loaded by the processor 801 and performs the following steps:

[0185] Based on the grouping value indicated by the filtering instruction information, the data analysis results of each user include a sequence of grouping values ​​for filtering processing, and S users are filtered out from at least one user, where S is an integer greater than or equal to 0.

[0186] The data analysis results of the S users are used as the first analysis result.

[0187] In one possible implementation, the data analysis request further includes conversion time interval indication information, and the computer program in the computer-readable storage medium 804 is further loaded by the processor 801 and performs the following steps:

[0188] Obtain the timestamp of each behavior step in the optimal conversion behavior sequence for each user;

[0189] The conversion time interval for each user is determined based on the timestamps of each behavioral step included in each user's behavior.

[0190] The shortest conversion time interval among all users is taken as the optimal conversion time interval.

[0191] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the data processing methods provided in the various alternative embodiments described above.

[0192] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0194] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0195] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: Receive a data analysis request, the data analysis request including data query conditions and analysis type indication information; Obtain user behavior data that matches the data query conditions, and integrate the user behavior data according to the user identifier corresponding to the user behavior data to obtain the behavior sequence data of each user; The behavioral sequence data of each user is analyzed and processed according to the analysis type indication information to obtain a first analysis result, which includes the data analysis result of each user.

2. The method according to claim 1, characterized in that, The step of performing data analysis processing on the behavioral sequence data of each user according to the analysis type indication information to obtain a first analysis result includes: Obtain the file address of the target data analysis module corresponding to the analysis type indication information; The module file of the target data analysis module is obtained from the file server using the file address; The target data analysis module is run according to its module file, and the behavioral sequence data of each user is analyzed and processed by the target data analysis module to obtain the first analysis result.

3. The method according to claim 2, characterized in that, The target data analysis module is created based on a function implementation template and registered to the file address based on a registration function. The target data analysis module includes any one of the following: funnel analysis module, retention analysis module, path analysis module, attribution analysis module, interval analysis module, lifetime value analysis module, and session analysis module.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: A data storage request is received, which is sent based on the matching result between a target hash and the identifiers of multiple task execution servers. The data storage request includes a first user identifier and user behavior data to be stored. The target hash is generated based on the first user identifier. In response to the data storage request, the first user identifier and the user behavior data to be stored are associated and stored in the local storage space.

5. The method according to any one of claims 1-3, characterized in that, The behavioral sequence data for each user includes N behavioral records arranged in chronological order, where N is an integer greater than or equal to 1. The step of performing data analysis processing on the behavioral sequence data for each user according to the analysis type indication information to obtain a first analysis result includes: When the analysis type indication information includes funnel analysis indication information, funnel analysis is performed on the N behavior records included in the behavior sequence data of each user in chronological order from earliest to latest timestamp to obtain the funnel set of each user; wherein, the funnel set includes M funnels, and the first behavior record included in each funnel corresponds to the initial behavior step of the funnel analysis, where M is the number of behavior records in the N behavior records that are located before the first behavior record and correspond to the initial behavior step; the first behavior record corresponds to the terminating behavior step of the funnel analysis, or the first behavior record is the last behavior record in the N behavior records; one of the funnels includes the first behavior record; The first analysis result is determined based on the behavioral records included in each of the M funnels for each user.

6. The method according to claim 5, characterized in that, The funnel analysis process is performed on the N behavior records of each user's behavior sequence data in chronological order from earliest to latest, to obtain the funnel set for each user, including: Query the first behavior record corresponding to the initial behavior step of the funnel analysis among the N behavior records included in the behavior sequence data of each user in the order of timestamp from early to late, generate the first funnel for each user, and add the first behavior record to the first funnel; The N behavior records are traversed in order of timestamp from earliest to latest, starting from the first behavior record. If the action step corresponding to the second action record being traversed is different from the initial action step, then the second action record is added to the first funnel; If the behavior step corresponding to the second behavior record is not the termination behavior step of the funnel analysis, then continue traversing until the behavior step corresponding to the currently traversed third behavior record is the termination behavior step, add the third behavior record to the first funnel, and stop traversing. Add the first funnel of each user to the funnel set of each user.

7. The method according to claim 6, characterized in that, The method further includes: If the behavior step corresponding to the second behavior record being traversed is the same as the initial behavior step, then a second funnel is generated for each user, and the second behavior record is added to the second funnel; The N behavior records are traversed in order of timestamp from earliest to latest, starting with the behavior record following the second behavior record. If the action step corresponding to the fourth row record being traversed is different from the initial action step, then the fourth row record is added to the second funnel. If the behavior step corresponding to the fourth behavior record is not the termination behavior step, then continue traversing until the behavior step corresponding to the currently traversed fifth behavior record is the termination behavior step, add the fifth behavior record to the second funnel, and stop traversing; Add the second funnel of each user to the funnel set of each user.

8. The method according to claim 5, characterized in that, The step of determining the first analysis result based on the behavioral records included in each of the M funnels for each user includes: Based on the behavior records in the M funnels included in each user's funnel set, and arranged in order from late to early timestamps, determine the optimal conversion behavior sequence for each user. Based on the grouping values ​​corresponding to the behavior records included in the optimal conversion behavior sequence of each user, determine the grouping value sequence for each user; The optimal conversion behavior sequence, grouping value sequence, and conversion depth for each user are used as the data analysis results for each user; wherein, the conversion depth is determined based on the number of behavioral steps included in the optimal conversion behavior sequence; The first analysis result is determined based on the data analysis results of each user.

9. The method according to claim 8, characterized in that, The step of determining the optimal conversion behavior sequence for each user based on the behavior records in the M funnels included in each user's funnel set, arranged in ascending order of timestamps, includes: The behavior records included in the M funnels of each user's funnel set are sorted according to the timestamp from earliest to latest to obtain L behavior records for each user. The last behavior record in the L behavior records is the behavior record corresponding to the termination behavior step, or the last behavior record in the L behavior records is the last behavior record in the N behavior records; where L is an integer greater than or equal to 0, and L is less than or equal to N. Obtain the corresponding I behavior records of each user's L behavior records within the update time window, where I is an integer greater than or equal to 0 and I is less than or equal to L; Query the I behavior records according to the timestamp from late to early and the behavior steps included in the funnel analysis, and determine the target behavior sequence that matches the behavior steps included in the funnel analysis; The target behavior sequence is taken as the optimal conversion behavior sequence for each user.

10. The method according to claim 8, characterized in that, The data analysis request also includes filtering instructions, and the method further includes: Based on the grouping value indicated by the filtering instruction information, the data analysis results of each user include a sequence of grouping values ​​for filtering processing, and S users are filtered out from at least one user, where S is an integer greater than or equal to 0. The data analysis results of the S users are used as the first analysis result.

11. The method according to claim 8, characterized in that, The data analysis request also includes conversion time interval indication information, and the method further includes: Obtain the timestamp of each behavior step in the optimal conversion behavior sequence for each user; The conversion time interval for each user is determined based on the timestamps of each behavioral step included in each user's behavior. The shortest conversion time interval among all users is taken as the optimal conversion time interval.

12. A data processing apparatus, characterized in that, The device includes: A transceiver unit is used to receive data analysis requests, which include data query conditions and analysis type indication information. The acquisition unit is used to acquire user behavior data that matches the data query conditions, and to integrate and process the user behavior data according to the user identifier corresponding to the user behavior data to obtain behavior sequence data for each user. The processing unit is configured to perform data analysis processing on the behavioral sequence data of each user according to the analysis type indication information to obtain a first analysis result, wherein the first analysis result includes the data analysis result of each user.

13. An electronic device, characterized in that, The electronic device includes: A processor, suitable for implementing one or more computer programs; and, A computer-readable storage medium storing one or more computer programs adapted to be loaded by the processor and executed as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more computer programs, which are adapted to be loaded by a processor and executed by the data processing method as described in any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the data processing method as described in any one of claims 1-11.

Images