Method and device for processing buried point data, computer device and storage medium

Abstract

The application relates to the field of big data, and provides a method and device for processing buried point data, computer equipment and a storage medium. The method comprises the following steps: obtaining original buried point data, performing theme division on the original buried point data based on a created buried point theme, obtaining buried point theme data, synchronizing the buried point theme data to a buried point theme table of a constructed distributed database, and performing data analysis based on the buried point theme data and the buried point theme table in the distributed database. The method can provide a unified data basis for subsequent data analysis, shield changes in buried point collection upgrading and reconstruction in the subsequent data analysis process, effectively reduce the influence of incompatibility between new and old buried point data on subsequent data analysis, and guarantee consistency of data analysis. In addition, theme division and data synchronization can be efficiently realized through design of the buried point theme and the buried point theme table of the distributed database, and development workload can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for processing embedded data. Background Technology

[0002] Mobile application data analytics is a typical big data processing scenario, following a unified big data processing workflow: data collection, preprocessing, statistical analysis, and data mining. In practice, mobile applications need to migrate to a unified enterprise data collection SDK (Software Development Kit) through version upgrades and updates. This results in the parallel collection of old and new data. To ensure the continuity of mobile application data analytics, a commonly used approach is to support data warehouse transformation while simultaneously modifying the application.

[0003] However, for application modification and support solutions, the main problem is that the data collected by different event tracking SDKs may be incompatible, which affects the consistency of subsequent data analysis and data mining processes, and thus affects the efficiency of data analysis.

[0004] Therefore, there is a need to provide a solution that can be compatible with data collected by different event tracking SDKs and ensure the consistency of mobile application data analysis. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for processing embedded data that can ensure the consistency of data analysis, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for processing embedded data. The method includes:

[0007] Obtain the original data from the embedded points;

[0008] Based on the created tracking topics, the original tracking data is divided into topics to obtain tracking topic data;

[0009] Synchronize the event tracking data to the event tracking table in the already built distributed database;

[0010] Data analysis is performed based on the event tracking data and the event tracking table in the distributed database.

[0011] In one embodiment, the created tracking themes include click themes, exposure themes, page themes, session and device themes, and page browsing themes;

[0012] Based on the created tracking topics, the original tracking data is divided into topics, resulting in tracking topic data including:

[0013] The raw event tracking data is mapped to at least one of the following event tracking themes: click theme, exposure theme, page theme, session and device theme, and page browsing theme, according to the preset mapping relationship between raw event tracking data and event tracking theme, to obtain event tracking theme data.

[0014] In one embodiment, the original event tracking data is divided into themes to obtain event tracking theme data, which includes:

[0015] The predefined big data cloud streaming computing tool is invoked to perform thematic segmentation of the raw event tracking data in a streaming computing manner, thereby obtaining event tracking theme data.

[0016] In one embodiment, synchronizing the event tracking topic data to the event tracking topic table of the already constructed distributed database includes:

[0017] The predefined data integration tool of the big data cloud is invoked to synchronize the event tracking data to the event tracking table of the built distributed database.

[0018] In one embodiment, invoking a predefined data integration tool from a big data cloud to synchronize the event tracking data to the event tracking table of a pre-built distributed database includes:

[0019] The predefined data integration tool of the big data cloud is invoked to synchronously map the event tracking data to the event tracking table of the built distributed database, based on the preset mapping relationship between the data source table and the event tracking theme table.

[0020] In one embodiment, data analysis based on event tracking topic data and event tracking topic tables in a distributed database includes:

[0021] Real-time calculation and analysis based on the data of the embedded points;

[0022] Offline calculation and analysis are performed based on the event tracking table in a distributed database.

[0023] Secondly, this application also provides a data processing device for embedded data. The device includes:

[0024] The data acquisition module is used to acquire raw embedded data.

[0025] The topic segmentation module is used to segment the original tracking data into topics based on the created tracking topics, so as to obtain tracking topic data;

[0026] The data synchronization module is used to synchronize the event tracking data to the event tracking table in the built distributed database.

[0027] The data analysis module is used to perform data analysis based on the event tracking data and the event tracking table in the distributed database.

[0028] In one embodiment, the created tracking themes include click themes, exposure themes, page themes, session and device themes, and page browsing themes;

[0029] The theme segmentation module is also used to map the raw tracking data to at least one of the following tracking themes: click theme, exposure theme, page theme, session and device theme, and page browsing theme, according to the preset mapping relationship between raw tracking data and tracking themes, to obtain tracking theme data.

[0030] In one embodiment, the topic segmentation module is also used to call a predefined big data cloud stream computing tool to perform topic segmentation on the original tracking data in a stream computing manner to obtain tracking topic data.

[0031] In one embodiment, the data synchronization module is also used to call a predefined data integration tool of a big data cloud to synchronize the event tracking data to the event tracking table of the built distributed database.

[0032] In one embodiment, the data synchronization module is further configured to invoke a predefined data integration tool of a big data cloud, and according to a preset mapping relationship between the data source table and the event tracking topic table, synchronize and map the event tracking topic data to the constructed distributed database.

[0033] In one embodiment, the data analysis module is also used to perform real-time calculation and analysis based on the event tracking data, and to perform offline calculation and analysis based on the event tracking table in the distributed database.

[0034] In addition to the predefined surface, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0035] Obtain the original data from the embedded points;

[0036] Based on the created tracking topics, the original tracking data is divided into topics to obtain tracking topic data;

[0037] Synchronize the event tracking data to the event tracking table in the already built distributed database;

[0038] Data analysis is performed based on the event tracking data and the event tracking table in the distributed database.

[0039] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0040] Obtain the original data from the embedded points;

[0041] Based on the created tracking topics, the original tracking data is divided into topics to obtain tracking topic data;

[0042] Synchronize the event tracking data to the event tracking table in the already built distributed database;

[0043] Data analysis is performed based on the event tracking data and the event tracking table in the distributed database.

[0044] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0045] Obtain the original data from the embedded points;

[0046] Based on the created tracking topics, the original tracking data is divided into topics to obtain tracking topic data;

[0047] Synchronize the event tracking data to the event tracking table in the already built distributed database;

[0048] Data analysis is performed based on the event tracking data and the event tracking table in the distributed database.

[0049] The aforementioned methods, devices, computer equipment, storage media, and computer program products for processing event tracking data do not directly transform the raw event tracking data after acquisition. Instead, they preprocess the raw event tracking data. This includes, on one hand, dividing the raw event tracking data into themes based on existing event tracking topics, resulting in event tracking topic data; and on the other hand, synchronizing the event tracking topic data to the event tracking topic table in a pre-built distributed database. Finally, data analysis is performed based on the event tracking topic data and the event tracking topic table in the distributed database. This solution, by preprocessing the raw event tracking data, splits the raw event tracking data (including data collected by both old and new SDKs) into unified themes and synchronizes the resulting event tracking topic data to the distributed database. This provides a unified data foundation for subsequent data analysis, shielding the data from changes in event tracking collection upgrades and effectively mitigating the impact of incompatibility between old and new event tracking data on subsequent data analysis. This ensures data analysis consistency and improves efficiency. Furthermore, the design of event tracking topics and the event tracking topic table in the distributed database efficiently achieves theme division and data synchronization while effectively reducing development workload. Attached Figure Description

[0050] Figure 1 This is an application environment diagram of the embedded data processing method in one embodiment;

[0051] Figure 2This is a flowchart illustrating a method for processing embedded data in one embodiment;

[0052] Figure 3 This is a flowchart illustrating the data processing method for embedded data in another embodiment;

[0053] Figure 4 This is a flowchart illustrating the data processing method for embedded data in another embodiment;

[0054] Figure 5 This is a detailed flowchart of the data processing method for embedded points in another embodiment;

[0055] Figure 6 This is a structural block diagram of the embedded data processing device in one embodiment;

[0056] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0059] The data processing method for embedded points provided in this application can be applied to, for example... Figure 1The application environment is shown. The data acquisition terminal 102 integrates a data tracking SDK and communicates with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104 or placed on a cloud or other network server. Specifically, the data acquisition SDK of the data acquisition terminal 102 can collect raw data tracking from various data sources and upload it to the acquisition gateway. Then, the server 104 obtains the raw data tracking, divides it into themes based on the created tracking themes, and obtains tracking theme data. Next, it synchronizes the tracking theme data to the tracking theme table of the built distributed database. Finally, it performs data analysis based on the tracking theme data and the tracking theme table in the distributed database. The data acquisition terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0060] In one embodiment, such as Figure 2 As shown, a method for processing embedded data is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0061] Step S202: Obtain the original embedded data.

[0062] A data source refers to the source of data, providing the location of the data required by the application. In practical applications, the data source can be pre-created by developers. Specifically, key configuration parameters of the data source include: data source name, data source type, test address, and production address. Data source types include relational databases, big data storage, message queues, and semi-structured storage. In this embodiment, the data source can be the distributed message queue Kafka. Raw event tracking data refers to the unprocessed raw event tracking data collected. Specifically, after obtaining authorization for data collection, the mobile application (APP) integrates a unified event tracking SDK resource package, used to modify event tracking according to specifications and upload event tracking data to the data collection gateway according to business needs. Raw event tracking data can be the interactive behavior data of objects within the mobile application.

[0063] Step S204: Based on the created tracking topics, the original tracking data is divided into topics to obtain tracking topic data.

[0064] A tracking topic, also known as a tracking theme, is the smallest unit for subscription and publication in a stream computing service. A topic represents a type of streaming data, similar to a table in a database. In practical applications, developers can pre-configure tracking topic parameters and create tracking topics. Tracking topics can include click topics and page topics, among others. Specifically, key configuration parameters for a tracking topic include the topic name, topic data format, number of topic partitions, and topic field settings. The topic data format includes JSON and CSV formats. Topic field settings include field name, type, length, and whether it is required. After developers pre-create the tracking topics, when they obtain raw tracking data, they can refer to the fields in the tracking topic's configuration parameters to partition the raw tracking data into corresponding pre-created tracking topics, thus obtaining the tracking topic data.

[0065] Step S206: Synchronize the event tracking data to the event tracking table of the built distributed database.

[0066] In practical applications, mobile application data analysis includes real-time computing and offline computing. To meet the needs of offline computing, developers can pre-create a distributed database and build event tracking topic tables within it. These topic tables correspond to specific topics, and their key fields are consistent with those in the topic tables. After obtaining the event tracking topic data, it can be synchronized to the event tracking topic tables in the pre-built distributed database to support subsequent offline metric calculations. In this embodiment, the distributed database can be a distributed parallel structured database cluster, specifically an offline MPP (Analytical Massively Parallel Processing (MPP) Databases library. It is understood that in other embodiments, other distributed databases such as Greenplum can be used, depending on the specific circumstances, and this is not limited here.

[0067] Step S208: Perform data analysis based on the event tracking data and the event tracking table in the distributed database.

[0068] Following the above embodiments, after obtaining the event tracking data and synchronizing it to the distributed database, data analysis can be performed based on the event tracking data and the event tracking table in the distributed database. The data analysis includes real-time calculation, offline calculation of indicator processing and analysis, and data mining.

[0069] In the above-described method for processing event tracking data, after acquiring the raw event tracking data, no direct data transformation is performed. Instead, the raw event tracking data undergoes preprocessing. This includes: firstly, dividing the raw event tracking data into themes based on existing event tracking topics to obtain event tracking topic data; and secondly, synchronizing the event tracking topic data to the event tracking topic table in the constructed distributed database. Finally, data analysis is performed based on the event tracking topic data and the event tracking topic table in the distributed database. This solution, by preprocessing the raw event tracking data, splits the raw event tracking data (including data collected by both old and new SDKs) into unified themes and synchronizes the resulting event tracking topic data to the distributed database. This provides a unified data foundation for subsequent data analysis, shielding the data from changes caused by upgrades and modifications in event tracking collection. This effectively mitigates the impact of incompatibility between old and new event tracking data on subsequent data analysis, ensuring consistency and improving efficiency. Furthermore, the design of event tracking topics and the event tracking topic table in the distributed database efficiently achieves theme division and data synchronization while also effectively reducing development workload.

[0070] like Figure 3 As shown, in one embodiment, the created tracking themes include click themes, exposure themes, page themes, session and device themes, and page browsing themes;

[0071] Step S204 includes: Step S224, mapping the original event tracking data to at least one event tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme according to a preset mapping relationship between original event tracking data and event tracking theme, to obtain event tracking theme data.

[0072] In this embodiment, the tracking themes include click themes, exposure themes, page themes, session and device themes, and page browsing themes. In practical applications, developers can abstract the raw tracking data into click themes, exposure themes, page themes, session and device themes, and page browsing themes, and pre-define the mapping relationship between the raw tracking data and the tracking themes. The exposure theme is used to count the number of times a specific module or area of ​​a page is effectively viewed. The core fields of the click theme include click type, click field, click ID (Identity), operation time, object session ID, device unique identifier, city code, and object number. The core fields of the exposure theme may include object type, operation type (display, claim), exposure name, exposure number, operation time, object session ID, device unique identifier, object number, and city code. The core fields of the page theme include page type, page number, operation time, object session ID, device unique identifier, city code, and object number. The core fields for session and device topics include session ID, page ID, login time, login method, device location information, operating system information, network type, device unique identifier, access channel, session start time, APP version information, and object ID. The core fields for page browsing topics include session ID, page ID, page name, page entry ID, page open time, page dwell time, operation time, open / exit flag, and object ID. Specifically, regarding the device unique identifier, for devices using the Android system, the collected device unique identifier is IMEI (International Mobile Equipment Identity), and for devices using the iOS system, the collected device unique identifier is UUID (Universally Unique Identifier).

[0073] In practical implementation, after obtaining the raw event tracking data, the raw event tracking data can be mapped to at least one of the following event tracking themes: click theme, exposure theme, page theme, session and device theme, and page browsing theme, according to a preset mapping relationship between raw event tracking data and event tracking themes, to obtain event tracking theme data. It is understood that the raw event tracking data can also be mapped to click theme, exposure theme, page theme, session and device theme, and page browsing theme separately. The specific mapping depends on the type of raw event tracking data and the actual situation. In this embodiment, by designing click theme, exposure theme, page theme, session and device theme, and page browsing theme, the raw event tracking data can be comprehensively and uniformly divided into corresponding event tracking themes, providing a unified basis for subsequent data analysis. The preset mapping relationship enables more efficient theme division of the raw event tracking data.

[0074] like Figure 4 As shown, in one embodiment, step S204 includes: step S244, calling a predefined big data cloud stream computing tool to perform topic division on the original tracking data in a stream computing manner to obtain tracking topic data.

[0075] In practical applications, the volume of raw event tracking data can be very large, involving both existing and newly added event tracking data. Therefore, the task of dividing the raw event tracking data into topics is quite complex and challenging. One approach is for developers to pre-create stream computing jobs within the stream computing component of a big data cloud platform, defining the stream computing business logic using SQL Operator. Specifically, the key configuration parameters for stream computing jobs include job name, data source, topic name, and Kafka consumption location. The Kafka consumption location includes three options: Latest (most recent time), Earliest (earliest time), and custom (custom start time selection).

[0076] In big data cloud services, a data source for one stream computing service corresponds to one topic, and a single topic can store one or more streaming data. In this embodiment, a predefined big data cloud stream computing tool can be invoked to perform topic partitioning of the original event tracking data in a stream computing manner, resulting in event tracking topic data. Specifically, according to the time of the original event tracking data, the original event tracking data is mapped to at least one event tracking topic among click topics, exposure topics, page topics, session and device topics, and page browsing topics, according to a preset mapping relationship between the original event tracking data and event tracking topics, to obtain event tracking topic data. In specific implementation, the event tracking topic data can be output to a Kafka sink. In this embodiment, by invoking a predefined big data cloud stream computing tool to perform topic partitioning of the original event tracking data in a stream computing manner, continuous and efficient topic partitioning can be achieved, greatly improving the efficiency of topic partitioning.

[0077] like Figure 3 As shown, in one embodiment, step S206 includes: step S226, calling a predefined big data cloud data integration tool to synchronize the event tracking data to the event tracking table of the built distributed database.

[0078] The big data cloud platform data integration tool is a suite of data processing tools that include data synchronization, data processing, and business verification. Following the above embodiment, to improve data synchronization efficiency, a data synchronization workflow can be pre-defined on the big data cloud platform data integration tool. By calling the pre-defined data synchronization workflow of the big data cloud data integration tool, the event tracking data can be synchronized to the event tracking table in the pre-built distributed database. Specifically, the event tracking data can be summarized daily and stored in the event tracking table in the distributed database to support subsequent offline metric calculations. In this embodiment, by calling the pre-defined big data cloud data integration tool to synchronize the event tracking data to the pre-built distributed database event tracking table, not only can development workload be reduced and data synchronization efficiency improved, but subsequent offline calculations can also be supported.

[0079] like Figure 4 As shown, in one embodiment, step S226 includes: step S246, calling a predefined big data cloud data integration tool, and synchronously mapping the event tracking data to the event tracking table of the constructed distributed database according to the preset mapping relationship between the data source table and the event tracking theme table.

[0080] Following the previous embodiment, the synchronization of event tracking data can be achieved by developers creating a data integration job in a predefined big data cloud platform. This mainly involves creating a data synchronization job, which includes the following steps:

[0081] Step 1: Select the data source table, i.e., the pre-created data source.

[0082] Step 2: Select the data target table, which is the pre-created event tracking table of the distributed database.

[0083] Step 3: Set up mapping management between the data source table and the data target table. Mapping management includes name-based mapping and manual configuration of mapping relationships to support subsequent synchronization of event tracking data. Specifically, during the mapping process, you can fill in field expressions for the source table fields, reassign values ​​to the data source table fields, or set default values ​​for the target table fields. When the source import is empty, the default values ​​will be used to assign values ​​to the target fields.

[0084] Following the above method, after creating the data integration job, a predefined data integration tool from a big data cloud can be invoked to synchronize the event tracking data (real-time Kafka data) to the data theme table in the distributed database, based on the preset mapping relationship between the data source table and the event tracking theme table. In this embodiment, accurate and efficient data synchronization can be achieved through the preset mapping relationship between the data source table and the event tracking theme table.

[0085] like Figure 3 As shown, in one embodiment, step S208 includes: step S228, performing real-time calculation and analysis based on the tracking point topic data, and performing offline calculation and analysis based on the tracking point topic table in the distributed database.

[0086] Since practical data analysis requires both real-time and offline computational analysis, real-time computational analysis is the primary focus. Therefore, in specific implementations, the event tracking data can be used as the data source for real-time computation, performing real-time computational analysis based on this data, specifically including real-time data metric calculation and data mining. The event tracking table in the distributed database can be used as the data source for offline computation, performing offline computational analysis based on this table, specifically including offline data metric calculation and data mining. In this embodiment, by using the event tracking data as the data foundation for real-time computation and the event tracking table in the distributed database as the data foundation for offline computation, it can flexibly meet the needs of data analysis and support efficient real-time and offline computational analysis.

[0087] To provide a clearer explanation of the data processing method for embedded points provided in this application, the following is in conjunction with the appendix. Figure 5 A specific embodiment will be described below, which includes the following steps:

[0088] Step S100: Obtain the original embedded data.

[0089] Step S200: Invoke the predefined big data cloud stream computing tool to map the raw event tracking data to at least one of the following event tracking themes: click theme, exposure theme, page theme, session and device theme, and page browsing theme, according to the preset mapping relationship between raw event tracking data and event tracking themes, thereby obtaining event tracking theme data. In specific implementation, the event tracking theme data can be output to a Kafka sink.

[0090] Step S300: Invoke the predefined data integration tool of the big data cloud, and synchronously map the event tracking data to the event tracking table of the built distributed database according to the preset mapping relationship between the data source table and the event tracking topic table. In specific implementation, the event tracking data in Kafka can be synchronously mapped to the event tracking table of the built MPP library.

[0091] Step S400: Perform real-time calculation and analysis based on the tracked data, and perform offline calculation and analysis based on the tracked data table in the distributed database.

[0092] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0093] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the aforementioned data processing method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the data processing apparatus provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0094] In one embodiment, such as Figure 6 As shown, a data processing device for embedded data points is provided, comprising: a data acquisition module 610, a topic segmentation module 620, a data synchronization module 630, and a data analysis module 640, wherein:

[0095] The data acquisition module 610 is used to acquire raw embedded data.

[0096] The topic segmentation module 620 is used to segment the original tracking data into topics based on the created tracking topics, so as to obtain tracking topic data.

[0097] The data synchronization module 630 is used to synchronize the tracking topic data to the tracking topic table of the built distributed database.

[0098] The data analysis module 640 is used for data analysis based on the event tracking data and the event tracking table in the distributed database.

[0099] The aforementioned event tracking data processing device, after acquiring the raw event tracking data, does not directly transform the data. Instead, it preprocesses the raw event tracking data. This includes, on one hand, dividing the raw event tracking data into themes based on existing event tracking topics, resulting in event tracking topic data; on the other hand, synchronizing the event tracking topic data to the event tracking topic table in the constructed distributed database; and finally, performing data analysis based on the event tracking topic data and the event tracking topic table in the distributed database. This solution, by preprocessing the raw event tracking data, splits the raw event tracking data (including data collected by both old and new SDKs) into unified themes and synchronizes the resulting event tracking topic data to the distributed database. This provides a unified data foundation for subsequent data analysis, shielding the data from changes in event tracking collection upgrades and effectively mitigating the impact of incompatibility between old and new event tracking data on subsequent data analysis. This ensures data analysis consistency and improves efficiency. Furthermore, the design of event tracking topics and the event tracking topic table in the distributed database efficiently achieves theme division and data synchronization, while also effectively reducing development workload.

[0100] In one embodiment, the created tracking themes include click themes, exposure themes, page themes, session and device themes, and page browsing themes;

[0101] The theme segmentation module 620 is also used to map the original tracking data to at least one of the following tracking themes: click theme, exposure theme, page theme, session and device theme, and page browsing theme, according to the preset mapping relationship between the original tracking data and the tracking theme, so as to obtain tracking theme data.

[0102] In one embodiment, the topic segmentation module 620 is also used to call a predefined big data cloud stream computing tool to perform topic segmentation on the original tracking data in a stream computing manner to obtain tracking topic data.

[0103] In one embodiment, the data synchronization module 630 is also used to call a predefined data integration tool of a big data cloud to synchronize the tracking topic data to the tracking topic table of the built distributed database.

[0104] In one embodiment, the data synchronization module 630 is further configured to invoke a predefined data integration tool of a big data cloud, and synchronize and map the tracking topic data to the constructed distributed database according to the preset mapping relationship between the data source table and the tracking topic table.

[0105] In one embodiment, the data analysis module 640 is also used to perform real-time calculation and analysis based on the tracking point theme data, and to perform offline calculation and analysis based on the tracking point theme table in the distributed database.

[0106] Each module in the aforementioned embedded data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0107] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores raw data points and related data. The I / O interfaces allow the processor to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements a data point processing method.

[0108] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0109] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring raw tracking data; dividing the raw tracking data into themes based on created tracking themes to obtain tracking theme data; synchronizing the tracking theme data to the tracking theme table of a constructed distributed database; and performing data analysis based on the tracking theme data and the tracking theme table in the distributed database.

[0110] In one embodiment, when the processor executes the computer program, it further performs the following steps: mapping the original tracking data to at least one tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme according to a preset mapping relationship between the original tracking data and the tracking theme, to obtain tracking theme data.

[0111] In one embodiment, when the processor executes the computer program, it also performs the following steps: calling a predefined big data cloud stream computing tool to perform topic division on the original tracking data in a stream computing manner to obtain tracking topic data.

[0112] In one embodiment, when the processor executes the computer program, it also performs the following steps: calling a predefined data integration tool of a big data cloud to synchronize the event tracking data to the event tracking table of the built distributed database.

[0113] In one embodiment, when the processor executes the computer program, it also performs the following steps: calling a predefined big data cloud data integration tool, and synchronously mapping the tracking point theme data to the constructed distributed database according to the preset mapping relationship between the data source table and the tracking point theme table.

[0114] In one embodiment, when the processor executes the computer program, it also performs the following steps: real-time calculation and analysis based on the tracked topic data, and offline calculation and analysis based on the tracked topic table in the distributed database.

[0115] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When the computer program is executed by a processor, it performs the following steps: acquiring raw tracking data; dividing the raw tracking data into themes based on the created tracking themes to obtain tracking theme data; synchronizing the tracking theme data to the tracking theme table of the constructed distributed database; and performing data analysis based on the tracking theme data and the tracking theme table in the distributed database.

[0116] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: mapping the original tracking data to at least one tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme according to a preset mapping relationship between the original tracking data and the tracking theme, to obtain tracking theme data.

[0117] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined big data cloud stream computing tool to perform topic division on the original tracking data in a stream computing manner to obtain tracking topic data.

[0118] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined data integration tool of a big data cloud to synchronize the event tracking data to the event tracking table of the built distributed database.

[0119] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined big data cloud data integration tool, and synchronously mapping the tracking topic data to the constructed distributed database according to the preset mapping relationship between the data source table and the tracking topic table.

[0120] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: real-time calculation and analysis based on the tracked topic data, and offline calculation and analysis based on the tracked topic table in the distributed database.

[0121] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring raw event tracking data; dividing the raw event tracking data into themes based on created event tracking themes to obtain event tracking theme data; synchronizing the event tracking theme data to the event tracking theme table of a constructed distributed database; and performing data analysis based on the event tracking theme data and the event tracking theme table in the distributed database.

[0122] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: mapping the original tracking data to at least one tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme according to a preset mapping relationship between the original tracking data and the tracking theme, to obtain tracking theme data.

[0123] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined big data cloud stream computing tool to perform topic division on the original tracking data in a stream computing manner to obtain tracking topic data.

[0124] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined data integration tool of a big data cloud to synchronize the event tracking data to the event tracking table of the built distributed database.

[0125] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: calling a predefined big data cloud data integration tool, and synchronously mapping the tracking topic data to the constructed distributed database according to the preset mapping relationship between the data source table and the tracking topic table.

[0126] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: real-time calculation and analysis based on the tracked topic data, and offline calculation and analysis based on the tracked topic table in the distributed database.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing embedded data, characterized in that, The method includes: Obtain raw event tracking data, which includes the interaction behavior data of the object in the mobile application; Based on the created tracking themes, the original tracking data is divided into themes to obtain tracking theme data; the step of dividing the original tracking data into themes to obtain tracking theme data includes: using stream computing, according to the time of the original tracking data, mapping the original tracking data to at least one tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme according to a preset mapping relationship between original tracking data and tracking themes, to obtain tracking theme data; The data tracking points are synchronized to the tracking point table in the existing distributed database. Data analysis is performed based on the aforementioned event tracking data and the event tracking table in the distributed database. The data analysis includes real-time calculation of metrics, processing and analysis of offline calculation metrics, and data mining.

2. The method according to claim 1, characterized in that, The process of dividing the original tracking data into thematic categories to obtain tracking theme data includes: The predefined big data cloud streaming computing tool is invoked to perform topic segmentation on the original data points in a streaming computing manner, thereby obtaining the data point topic data.

3. The method according to claim 1, characterized in that, The step of synchronizing the event tracking data to the event tracking table of the constructed distributed database includes: The predefined data integration tool of the big data cloud is invoked to synchronize the data of the event tracking topic to the event tracking topic table of the already constructed distributed database.

4. The method according to claim 3, characterized in that, The step of calling a predefined big data cloud data integration tool to synchronize the event tracking data to the event tracking table of the already constructed distributed database includes: The predefined data integration tool of the big data cloud is invoked to synchronously map the data of the data tracking topic to the data tracking topic table of the built distributed database, according to the preset mapping relationship between the data source table and the tracking topic table.

5. The method according to claim 1, characterized in that, The data analysis based on the event tracking data and the event tracking table in the distributed database includes: Real-time calculation and analysis are performed based on the aforementioned embedded data. Offline calculation and analysis are performed based on the tracking topic table in the distributed database.

6. A device for processing embedded data, characterized in that, The device includes: The data acquisition module is used to acquire raw tracking data, which includes the interaction behavior data of the object in the mobile application. The theme segmentation module is used to map the original event tracking data to at least one event tracking theme among click theme, exposure theme, page theme, session and device theme, and page browsing theme in a stream computing manner according to the time of the original event tracking data and according to the preset mapping relationship between the original event tracking data and the event tracking theme, so as to obtain event tracking theme data. The data synchronization module is used to synchronize the data of the tracking points to the tracking point table of the constructed distributed database; The data analysis module is used to perform data analysis based on the event tracking data and the event tracking table in the distributed database. The data analysis includes real-time calculation of indicators, processing and analysis of offline calculation indicators, and data mining.

7. The apparatus according to claim 6, characterized in that, The topic segmentation module is also used to call the predefined big data cloud stream computing tool to perform topic segmentation on the original tracking data in a stream computing manner to obtain tracking topic data.

8. The apparatus according to claim 6, characterized in that, The data synchronization module is also used to call the predefined data integration tool of the big data cloud to synchronize the data tracking topic data to the data tracking topic table of the constructed distributed database.

9. The apparatus according to claim 8, characterized in that, The data synchronization module is also used to call the predefined data integration tool of the big data cloud, and according to the preset mapping relationship between the data source table and the tracking topic table, synchronize and map the tracking topic data to the tracking topic table of the constructed distributed database.

10. The apparatus according to claim 6, characterized in that, The data analysis module is also used to perform real-time calculation and analysis based on the data tracking points, and to perform offline calculation and analysis based on the data tracking point table in the distributed database.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images