User behavior path analysis methods, devices, equipment, and storage media

Abstract

This invention discloses a user behavior path analysis method, apparatus, device, and storage medium. The method includes: acquiring user operation tracking data and storing the operation tracking data in a data warehouse to obtain operation tracking logs; consuming and processing the operation tracking data through a Spark Streaming application, and associating it with user ID information, user branch information, and product dimension tables to obtain a real-time table; using a Spark scheduled task to acquire the operation tracking logs, and associating the operation tracking logs with user ID information, user branch information, and product dimension tables to obtain an offline table; and analyzing user behavior paths based on the real-time table and the offline table. Compared with existing technologies, this method not only improves the accuracy of user behavior path analysis but also effectively enhances the timeliness of data in user behavior path analysis.

Description

Technical Field

[0001] This invention relates to the field of data query technology, and in particular to a user behavior path analysis method, apparatus, device, and storage medium. Background Technology

[0002] User behavior path analysis is an analytical method that helps understand user flow and, consequently, how users use an application. It primarily relies on user behavior logs to analyze the patterns and characteristics of user movement across various events, thereby enabling specific business applications.

[0003] To meet the operational staff's need for post-product launch performance review and improve digital operational capabilities, it is necessary to analyze user behavior paths based on the event tracking logs of the mobile banking app. However, the sheer volume of event tracking data, the variety of analysis scenarios and formats, and the increasingly stringent timeliness requirements necessitate continuous innovation in user behavior path analysis systems to meet these higher demands and more severe challenges. While user behavior path analysis can achieve offline user behavior queries and traffic analysis in certain scenarios, it is not the only solution.

[0004] However, the following shortcomings exist: Spark applications can only meet the data query requirements of the platform on T+2 days due to the large amount of data, complex path connections, and many fields in the result table, and cannot meet the business needs with real-time requirements when reading and processing more than 1 billion offline tracking points and user / product dimension table data on T+1 days.

[0005] Therefore, there is an urgent need for a user behavior path analysis method that solves the technical problem of low data timeliness in existing user behavior path analysis technologies. Summary of the Invention

[0006] The main objective of this invention is to provide a user behavior path analysis method, apparatus, device, and storage medium, aiming to solve the technical problem of low data timeliness in existing user behavior path analysis technologies.

[0007] To achieve the above objectives, the present invention provides a user behavior path analysis method, the method comprising the following steps:

[0008] Obtain user operation tracking data and store the operation tracking data in a data warehouse to obtain operation tracking logs;

[0009] The operation tracking data is consumed and processed through Spark Streaming application, and associated with user ID information, user branch information and product dimension table to obtain real-time table;

[0010] The operation tracking logs are obtained using a Spark scheduled task, and the operation tracking logs are associated with the user ID information, the user branch information and the product dimension table to obtain an offline table;

[0011] User behavior paths are analyzed based on the real-time table and the offline table.

[0012] Optionally, the step of consuming and processing the operation tracking data through a Spark Streaming application and associating it with user ID information, user branch information, and product dimension table to obtain a real-time table includes:

[0013] Acquire user-generated operation tracking data and push it to the Kafka cluster in real time;

[0014] The Spark Streaming application performs logical business processing on the operation tracking data of the Kafka cluster, and performs filtering, deduplication, classification and aggregation operations on the operation tracking data according to preset requirements to obtain the target data fields.

[0015] After associating the target field with user ID information, user branch information, and product dimension table, the data is written into the database to obtain a real-time table.

[0016] Optionally, the step of obtaining user operation tracking data and storing the operation tracking data in a data warehouse to obtain operation tracking logs includes:

[0017] Acquire user-generated operation tracking data and push it to the Kafka cluster in real time to form a Kafka data queue;

[0018] By performing data ETL processing on the Kafka data queue, the operation tracking data is loaded into the data warehouse to obtain operation tracking logs.

[0019] Optionally, the step of using a Spark scheduled task to obtain the operation tracking logs and associating the operation tracking logs with the user ID information, the user row information, and the product dimension table to obtain an offline table includes:

[0020] A Spark scheduled task is used to read the operation tracking logs, user ID information, user branch information, and product dimension table from the data warehouse.

[0021] After associating the operation tracking logs with the user ID information, the user branch information, and the product dimension table, the logs are written to the database to obtain an offline table.

[0022] Optionally, before the step of analyzing user behavior paths based on the real-time table and the offline table, the method further includes:

[0023] Obtain the query request from the target user;

[0024] Based on the query request, the query request information of the target user is determined, and the query request information includes the target tracking point identifier and the target query conditions.

[0025] Optionally, the database storing the real-time and offline tables is a column-oriented analytical database, and the step of analyzing user behavior paths based on the real-time and offline tables includes:

[0026] The query analysis statement is optimized based on the funnel function of the analytical database and the query request information to obtain an optimized query analysis statement;

[0027] Based on the real-time table and the offline table, the user behavior path is analyzed using the optimized query analysis statement.

[0028] Optionally, the operation tracking data includes operation tracking data of users on various pages and function columns in the network platform. The operation tracking data includes operation tracking identifiers, which include page identifiers and function column identifiers.

[0029] Furthermore, to achieve the above objectives, the present invention also proposes a user behavior path analysis device, the device comprising:

[0030] The data storage module is used to acquire user operation tracking data and store the operation tracking data in a data warehouse to obtain operation tracking logs.

[0031] The real-time table creation module is used to consume and process the operation tracking data through the Spark Streaming application, and associate the operation tracking data with user ID information, user branch information and product dimension table to obtain a real-time table;

[0032] The offline table creation module is used to obtain the operation tracking logs using a Spark scheduled task, and associate the operation tracking logs with the user ID information, the user row information and the product dimension table to obtain an offline table;

[0033] The path analysis module is used to analyze user behavior paths based on the real-time table and the offline table.

[0034] Furthermore, to achieve the above objectives, the present invention also proposes a user behavior path analysis device, the device comprising: a memory, a processor, and a user behavior path analysis program stored in the memory and executable on the processor, the user behavior path analysis program being configured to implement the steps of the user behavior path analysis method as described above.

[0035] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a user behavior path analysis program, which, when executed by a processor, implements the steps of the user behavior path analysis method as described above.

[0036] This invention obtains user operation tracking data and stores it in a data warehouse to obtain operation tracking logs. It then consumes and processes this data using a Spark Streaming application, and associates it with user ID information, user branch information, and product dimension tables to obtain a real-time table. A Spark scheduled task is used to retrieve the operation tracking logs and associate them with the user ID information, user branch information, and product dimension tables to obtain an offline table. User behavior paths are then analyzed based on the real-time and offline tables. Because this invention consumes and processes the operation tracking data using a Spark Streaming application and writes it to a database in real time to obtain a real-time table, and then uses a Spark scheduled task to retrieve the operation tracking logs and associate them with user ID information, user branch information, and product dimension tables to obtain an offline table, and then analyzes user behavior paths based on both the real-time and offline tables, this invention not only improves the accuracy of user behavior path analysis compared to existing technologies but also effectively improves the timeliness of the data. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of the user behavior path analysis device in the hardware operating environment involved in the embodiments of the present invention;

[0038] Figure 2 This is a flowchart illustrating the first embodiment of the user behavior path analysis method of the present invention;

[0039] Figure 3 This is a flowchart illustrating the second embodiment of the user behavior path analysis method of the present invention;

[0040] Figure 4 This is a flowchart illustrating the third embodiment of the user behavior path analysis method of the present invention.

[0041] Figure 5 This is a structural block diagram of the first embodiment of the user behavior path analysis device of the present invention.

[0042] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0043] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0044] Reference Figure 1 , Figure 1 This is a schematic diagram of the user behavior path analysis device structure in the hardware operating environment involved in the embodiments of the present invention.

[0045] like Figure 1 As shown, the user behavior path analysis device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0046] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the user behavior path analysis device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0047] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a user behavior path analysis program.

[0048] exist Figure 1In the user behavior path analysis device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the user behavior path analysis device of the present invention can be set in the user behavior path analysis device, and the user behavior path analysis device calls the user behavior path analysis program stored in the memory 1005 through the processor 1001 and executes the user behavior path analysis method provided in the embodiment of the present invention.

[0049] This invention provides a user behavior path analysis method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the user behavior path analysis method of the present invention.

[0050] In this embodiment, the user behavior path analysis method includes the following steps:

[0051] Step S10: Obtain the user's operation tracking data and store the operation tracking data in the data warehouse to obtain the operation tracking log.

[0052] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer or personal computer, or an electronic device or user behavior path analysis device capable of performing the above functions. The following uses a user behavior path analysis device as an example to illustrate this embodiment and the subsequent embodiments.

[0053] Understandably, user action tracking data can be records of user actions during the loading of pages and the clicking of function bars while using a mobile banking app. This action tracking data includes user action data across various pages and function bars on a web platform (e.g., a mobile banking app). The action tracking data includes action tracking identifiers, which consist of a page identifier and a function bar identifier. A page contains at least one function bar, and each page and function bar has its own unique identifier.

[0054] In a practical implementation, user-generated operation tracking data can be acquired and pushed to the Kafka cluster in real time to form a Kafka data queue. By performing data ETL processing on the Kafka data queue, the operation tracking data can be loaded into the data warehouse to obtain operation tracking logs.

[0055] Specifically, user behavior monitoring points are set at various locations within the mobile banking app. These multiple user behavior monitoring points are then used to monitor the user's operation records on the mobile banking app. In order to ensure a complete reconstruction of the user's behavior path, in this embodiment, the operation tracking data includes records of the user loading pages and clicking on function bars on the pages. These behavior records are stored in a data warehouse to obtain the operation tracking log.

[0056] Step S20: Consume the operation tracking data through the Spark Streaming application, and associate it with user ID information, user branch information and product dimension table to obtain a real-time table.

[0057] It's worth noting that Spark Streaming is an extension of the Spark core API, offering high throughput, strong fault tolerance, and real-time data streaming. It supports data sources including Kafka, Flume, HDFS / S3, and TCP Sockets. After acquiring data, users can perform complex algorithmic processing using advanced functions such as Map, Reduce, Join, and Window. The processing results can be stored in the file system, database, or displayed on dashboards. Furthermore, Spark Streaming can be integrated with other Spark components, such as MLlib and GraphX, to perform even more complex processing on real-time data.

[0058] In the specific implementation, Spark Streaming is used to consume the data from the Kafka tracking system in real time, and then associate it with the user ID information, user branch information and product dimension tables in the data warehouse. The resulting data table is then written to the database to obtain the real-time table.

[0059] It should be understood that, based on offline computing, a new Spark Streaming application is added to consume and process real-time data from Kafka's embedded points, writing the data to the database in real time. Compared to offline data, this significantly shortens the data chain and reduces the risk of data loss. It enables real-time queries for scenario attribution analysis and T+2 hour query timeliness for traffic analysis. This helps operations personnel to review business performance within T+2 hours.

[0060] Step S30: Use a Spark scheduled task to obtain the operation tracking logs, and associate the operation tracking logs with the user ID information, the user branch information and the product dimension table to obtain an offline table.

[0061] It should be explained that since the user ID information, user branch information, and product dimension table associated with the real-time data entry operation tracking data are T+1 offline data, the offline table is still retained to ensure the accuracy of the final data. This offline table, along with the real-time table, provides path analysis capabilities. Furthermore, the real-time table has the same structure as the offline table, providing real-time scenario analysis capabilities and T+2 hour near real-time traffic analysis capabilities, helping operations personnel improve the timeliness of business operation performance review.

[0062] It should be noted that Spark tasks can use sparkStreamingContext to set the execution interval and execute scheduled tasks.

[0063] It should be noted that a Spark scheduled task can be used to read the operation tracking logs, user ID information, user row information, and product dimension table from the data warehouse; after associating the operation tracking logs with the user ID information, user row information, and product dimension table, the data warehouse data can be written to the database to obtain an offline table.

[0064] Specifically, offline data is imported into the database by reading the operation logs of the data warehouse that have been imported into Hive on the T+1 day using a Spark task. After obtaining the data table by associating user ID information, user branch information and product dimension tables for various scenarios, the data is stored in the database to obtain an offline table for the device to query on the T+2 day.

[0065] Step S40: Analyze user behavior paths based on the real-time table and the offline table.

[0066] It should be understood that before step S40, the method further includes: obtaining the query request of the target user; determining the query request information of the target user based on the query request, wherein the query request information includes the target tracking point identifier and the target query conditions.

[0067] It should be noted that the target query conditions include filtering conditions such as time range, starting point type, ending point type, and specific content to be displayed, thereby enabling user behavior path query and fully meeting various query needs.

[0068] This embodiment acquires user-generated operation tracking data and pushes it to a Kafka cluster in real time, forming a Kafka data queue. The operation tracking data is then loaded into a data warehouse through ETL processing of the Kafka data queue to obtain operation tracking logs. The operation tracking data is consumed and processed using a Spark Streaming application, and associated with user ID information, user branch information, and product dimension tables to obtain a real-time table. A Spark scheduled task is used to acquire the operation tracking logs, and these logs are associated with the user ID information, user branch information, and product dimension tables to obtain an offline table. User behavior paths are then analyzed based on the real-time and offline tables. Because this invention consumes and processes the operation tracking data using a Spark Streaming application and writes it to a database in real time to obtain a real-time table, and then uses a Spark scheduled task to acquire the operation tracking logs and associate them with user ID information, user branch information, and product dimension tables to obtain an offline table, and then analyzes user behavior paths based on the real-time and offline tables, this invention not only improves the accuracy of user behavior path analysis compared to existing technologies but also effectively improves the timeliness of user behavior path analysis data.

[0069] refer to Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the user behavior path analysis method of the present invention.

[0070] Based on the first embodiment described above, in this embodiment, step S20 includes:

[0071] Step S201: Obtain the operation tracking data generated by the user and push it to the Kafka cluster in real time.

[0072] It's important to note that Kafka is a high-throughput distributed publish-subscribe messaging system (message engine system) that can handle all user action streams on a website. These actions (web browsing, searching, and other user actions) are a key element of many social functions on the modern web. This data is typically handled through log processing and log aggregation due to throughput requirements. For log data and offline analysis systems like Hadoop, but with the constraint of requiring real-time processing, Kafka is a viable solution. Kafka aims to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time messaging through clustering.

[0073] Step S202: Perform logical business processing on the operation tracking data of the Kafka cluster through the Spark Streaming application. According to preset requirements, perform filtering, deduplication, classification and aggregation operations on the operation tracking data to obtain the target data fields.

[0074] Step S203: After associating the target field with user ID information, user branch information and product dimension table, write it into the database to obtain a real-time table.

[0075] It should be noted that this embodiment does not require path concatenation calculations for a specific scenario, nor does it require storing intermediate results. Instead, it extracts the target data fields and simply associates them with user ID information, user branch information, and product dimension tables to form the final query data. Based on this data table, various business scenario requirements can be met simply by writing different SQL statements, including traffic analysis, funnel analysis, traffic conversion, attribution analysis, etc., and it can also adapt to various deeply customized scenario analyses.

[0076] This embodiment acquires user-generated operation tracking data and pushes it to a Kafka cluster in real time. The Spark Streaming application performs logical business processing on the operation tracking data in the Kafka cluster, filtering, deduplicating, classifying, and aggregating the data according to preset requirements to obtain target data fields. These target fields are then associated with user ID information, user line information, and product dimension tables, and written to a database to obtain a real-time table. Finally, user behavior paths are analyzed based on the real-time and offline tables. Compared to existing technologies, this invention not only improves the accuracy of user behavior paths but also effectively enhances the timeliness of user behavior path analysis data.

[0077] refer to Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the user behavior path analysis method of the present invention.

[0078] Based on the above embodiments, in this embodiment, step S40 includes:

[0079] Step S401: Optimize the query analysis statement based on the funnel function of the analytical database and the query request information to obtain the optimized query analysis statement.

[0080] It should be noted that the database used to store the real-time and offline tables in this invention is a column-oriented analytical database, ClickHouse(CK).

[0081] Understandably, the analytical database ClickHouse (CK) has a high compression ratio, and due to the optimization of the underlying data storage structure and query, it does not need to store intermediate results. The extracted target data fields are simply associated with user ID information, user row information and product dimension tables to form the final query data table, which greatly reduces the data storage space.

[0082] Step S402: Analyze user behavior paths based on the real-time table and the offline table using the optimized query analysis statement.

[0083] It's worth noting that the analytical database ClickHouse (CK) boasts high query performance, capable of processing hundreds of millions or even billions of rows of data per second per server. Furthermore, it fully leverages funnel functions, enabling various analytical modes such as traffic analysis, funnel analysis, traffic conversion, and attribution analysis, as well as customized business analysis models, through the writing and optimization of SQL statements, thereby enhancing the user query experience.

[0084] In this embodiment, the database storing the real-time and offline tables is a column-oriented analytical database. The query analysis statement is optimized based on the funnel function of the analytical database and the query request information to obtain an optimized query analysis statement. User behavior paths are analyzed based on the optimized query analysis statement using the real-time and offline tables. Compared to existing technologies, this not only reduces storage space and improves query efficiency, but also allows for the writing of different SQL statements to meet the needs of various business scenarios, including traffic analysis, funnel analysis, traffic conversion, and attribution analysis, and can adapt to various deeply customized scenario analyses.

[0085] Furthermore, this embodiment of the invention also proposes a storage medium storing a user behavior path analysis program, which, when executed by a processor, implements the steps of the user behavior path analysis method described above.

[0086] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the user behavior path analysis device of the present invention.

[0087] like Figure 5 As shown, the user behavior path analysis device proposed in this embodiment of the invention includes: a data storage module 501, a real-time table creation module 502, an offline table creation module 503, and a path analysis module 504.

[0088] The data storage module 501 is used to acquire the user's operation tracking data and store the operation tracking data in a data warehouse to obtain operation tracking logs.

[0089] The real-time table creation module 502 is used to consume and process the operation tracking data through the Spark Streaming application, and associate the operation tracking data with user ID information, user branch information and product dimension table to obtain a real-time table.

[0090] The offline table creation module 503 is used to obtain the operation tracking logs using a Spark scheduled task, and associate the operation tracking logs with the user ID information, the user branch information and the product dimension table to obtain an offline table.

[0091] The path analysis module 504 is used to analyze user behavior paths based on the real-time table and the offline table.

[0092] The data storage module 501 is also used to acquire user-generated operation tracking data and push it to the Kafka cluster in real time to form a Kafka data queue; by performing data ETL processing on the Kafka data queue, the operation tracking data is loaded into the data warehouse to obtain operation tracking logs.

[0093] The offline table creation module 503 is further configured to use a Spark scheduled task to read the operation tracking logs in the data warehouse, as well as the user ID information, user row information, and product dimension table in the data warehouse; and after associating the operation tracking logs with the user ID information, user row information, and product dimension table, write them into the database to obtain the offline table.

[0094] The path analysis module 504 is also used to obtain the query request of the target user; and to determine the query request information of the target user based on the query request, wherein the query request information includes the target tracking point identifier and the target query conditions.

[0095] This embodiment acquires user-generated operation tracking data and pushes it to a Kafka cluster in real time, forming a Kafka data queue. The operation tracking data is then loaded into a data warehouse through ETL processing of the Kafka data queue to obtain operation tracking logs. The operation tracking data is consumed and processed using a Spark Streaming application, and associated with user ID information, user branch information, and product dimension tables to obtain a real-time table. A Spark scheduled task is used to acquire the operation tracking logs, and these logs are associated with the user ID information, user branch information, and product dimension tables to obtain an offline table. User behavior paths are then analyzed based on the real-time and offline tables. Because this invention consumes and processes the operation tracking data using a Spark Streaming application and writes it to a database in real time to obtain a real-time table, and then uses a Spark scheduled task to acquire the operation tracking logs and associate them with user ID information, user branch information, and product dimension tables to obtain an offline table, and then analyzes user behavior paths based on the real-time and offline tables, this invention not only improves the accuracy of user behavior path analysis compared to existing technologies but also effectively improves the timeliness of user behavior path analysis data.

[0096] Based on the first embodiment of the user behavior path analysis device of the present invention described above, a second embodiment of the user behavior path analysis device of the present invention is proposed.

[0097] In this embodiment, the real-time table creation module 502 is further configured to acquire user-generated operation tracking data and push it to the Kafka cluster in real time; perform logical business processing on the operation tracking data of the Kafka cluster through the Spark Streaming application, and perform filtering, deduplication, classification and aggregation operations on the operation tracking data according to preset requirements to obtain target data fields; after associating the target fields with user ID information, user row information and product dimension table, write them into the database to obtain a real-time table.

[0098] The path analysis module 504 is further configured to store the real-time table and the offline table in a column-oriented analytical database; optimize the query analysis statement based on the funnel function of the analytical database and the query request information to obtain an optimized query analysis statement; and analyze the user behavior path based on the real-time table and the offline table using the optimized query analysis statement.

[0099] Other embodiments or specific implementations of the user behavior path analysis device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0100] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0101] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0103] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A user behavior path analysis method, characterized in that, The user behavior path analysis method includes the following steps: Obtain user operation tracking data and store the operation tracking data in a data warehouse to obtain operation tracking logs; The operation tracking data is consumed and processed through Spark Streaming application, and associated with user ID information, user branch information and product dimension table to obtain real-time table; The operation tracking logs are obtained using a Spark scheduled task, and the operation tracking logs are associated with the user ID information, the user branch information and the product dimension table to obtain an offline table; User behavior paths are analyzed based on the real-time table and the offline table; The steps of consuming and processing the operation tracking data through a Spark Streaming application, and associating it with user ID information, user branch information, and product dimension table to obtain a real-time table include: Acquire user-generated operation tracking data and push it to the Kafka cluster in real time; The Spark Streaming application performs logical business processing on the operation tracking data of the Kafka cluster, and performs filtering, deduplication, classification and aggregation operations on the operation tracking data according to preset requirements to obtain the target data fields. After associating the target data fields with user ID information, user branch information, and product dimension table, the data is written into the database to obtain a real-time table. The step of using a Spark scheduled task to obtain the operation tracking logs and associating the operation tracking logs with the user ID information, the user branch information, and the product dimension table to obtain an offline table includes: A Spark scheduled task is used to read the operation tracking logs, user ID information, user branch information, and product dimension table from the data warehouse. After associating the operation tracking logs with the user ID information, the user branch information, and the product dimension table, the logs are written to the database to obtain an offline table.

2. The user behavior path analysis method as described in claim 1, characterized in that, The steps of obtaining user operation tracking data and storing the operation tracking data in a data warehouse to obtain operation tracking logs include: Acquire user-generated operation tracking data and push it to the Kafka cluster in real time to form a Kafka data queue; By performing data ETL processing on the Kafka data queue, the operation tracking data is loaded into the data warehouse to obtain operation tracking logs.

3. The user behavior path analysis method as described in claim 1, characterized in that, Before the step of analyzing user behavior paths based on the real-time table and the offline table, the method further includes: Obtain the query request from the target user; Based on the query request, the query request information of the target user is determined, and the query request information includes the target tracking point identifier and the target query conditions.

4. The user behavior path analysis method as described in claim 3, characterized in that, The database storing the real-time and offline tables is a column-oriented analytical database. The step of analyzing user behavior paths based on the real-time and offline tables includes: The query analysis statement is optimized based on the funnel function of the analytical database and the query request information to obtain an optimized query analysis statement; Based on the real-time table and the offline table, the user behavior path is analyzed using the optimized query analysis statement.

5. The user behavior path analysis method as described in any one of claims 1-4, characterized in that, The operation tracking data includes operation tracking data of users on various pages and function columns in the network platform. The operation tracking data includes operation tracking identifiers, which include page identifiers and function column identifiers.

6. A user behavior path analysis device, characterized in that, The user behavior path analysis device includes: The data storage module is used to acquire user operation tracking data and store the operation tracking data in a data warehouse to obtain operation tracking logs. The real-time table creation module is used to consume and process the operation tracking data through the Spark Streaming application, and associate the operation tracking data with user ID information, user branch information and product dimension table to obtain a real-time table; The offline table creation module is used to obtain the operation tracking logs using a Spark scheduled task, and associate the operation tracking logs with the user ID information, the user row information and the product dimension table to obtain an offline table; The path analysis module is used to analyze user behavior paths based on the real-time table and the offline table; The real-time table creation module is also used to acquire user-generated operation tracking data and push it to the Kafka cluster in real time; the Spark Streaming application performs logical business processing on the operation tracking data in the Kafka cluster, and performs filtering, deduplication, classification and aggregation operations on the operation tracking data according to preset requirements to obtain target data fields; after associating the target data fields with user ID information, user row information and product dimension table, it writes them into the database to obtain a real-time table; The offline table creation module is further configured to use a Spark scheduled task to read the operation tracking logs in the data warehouse, as well as the user ID information, user row information, and product dimension table in the data warehouse; associate the operation tracking logs with the user ID information, user row information, and product dimension table, and then write them into the database to obtain the offline table.

7. A user behavior path analysis device, characterized in that, The device includes: a memory, a processor, and a user behavior path analysis program stored in the memory and executable on the processor, the user behavior path analysis program being configured to implement the steps of the user behavior path analysis method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a user behavior path analysis program, which, when executed by a processor, implements the steps of the user behavior path analysis method as described in any one of claims 1 to 5.

Images