Data analysis method and device, system, storage medium, and electronic device

Abstract

The present disclosure relates to the field of data processing technology, and specifically to a data analysis method and device, system, storage medium and electronic device. The method includes: obtaining original activity data of the target time, analyzing the original activity data to extract the active user metadata set corresponding to the active reached users; performing feature extraction on the active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features; based on historical user feature data, combining the user metadata set and the active user feature metadata set to perform feature aggregation to obtain an active user temporal feature metadata set containing user temporal behavior features; using preset dimension information to process the active user temporal feature metadata set to obtain primary user feature data containing preset dimensions. The primary user feature data obtained by this solution is convenient for mining target potential users again.

Description

Background Art

[0002] With the rapid development of Internet technology, the scale of various e-commerce companies is constantly expanding; at the same time, the operational data of e-commerce is also expanding. It is becoming increasingly important to obtain data on marketing activities, user shopping intentions, etc. through effective and accurate analysis of operational data.

[0003] Most of the existing technologies use offline computing to calculate the relevant data of activity effects, and obtain the statistical results of user behavior by roughly counting the data. However, this method cannot be effectively expanded according to business needs and system load.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a data analysis method, a data analysis device, a data analysis system, a storage medium and an electronic device, thereby overcoming the limitations and defects of the related art at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to a first aspect of the present disclosure, there is provided a data analysis method, comprising:

[0008] Obtaining original activity data at a target time, and analyzing the original activity data to extract an activity user metadata set corresponding to the activity-reached users;

[0009] Extracting features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features;

[0010] Based on the historical user feature data, feature aggregation is performed in combination with the user metadata set and the active user feature metadata set to obtain an active user temporal feature metadata set containing user temporal behavior features;

[0011] The active user temporal feature metadata set is processed using preset dimension information to obtain primary user feature data containing preset dimensions.

[0012] In an exemplary embodiment of the present disclosure, the method further includes:

[0013] Report data is generated based on the primary user feature data and stored in a search engine.

[0014] In an exemplary embodiment of the present disclosure, the method further includes:

[0015] In response to the target feature input by the user, the primary user feature data is screened using the target feature to obtain the secondary target user.

[0016] In an exemplary embodiment of the present disclosure, the obtaining of original activity data includes:

[0017] In response to the activity trigger instruction, a pending task is created;

[0018] Based on the task to be processed, a target service is called to obtain the raw activity data.

[0019] In an exemplary embodiment of the present disclosure, extracting features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features includes:

[0020] Feature extraction is performed on the active users in the active user metadata set based on preset behavior features to screen users containing the preset behavior features and generate the active user feature metadata set.

[0021] In an exemplary embodiment of the present disclosure, the feature aggregation is performed based on the historical user feature data in combination with the user metadata set and the active user feature metadata set to obtain an active user temporal feature metadata set containing user temporal behavior features, including:

[0022] Classifying users in the user metadata set to obtain first-type users and second-type users corresponding to the current target time;

[0023] Based on the first type of users corresponding to the target feature and the active user feature metadata set corresponding to the target feature in the previous time, updating the first type of user sequence corresponding to the target feature; and

[0024] Based on the second type of users, the active user temporal feature metadata set corresponding to the previous time is updated to obtain a second type of user sequence corresponding to the updated target feature;

[0025] The active user temporal feature metadata set corresponding to the target feature within the current target time is constructed based on the updated first type user sequence and the second type user sequence.

[0026] In an exemplary embodiment of the present disclosure, the method further includes:

[0027] The active user temporal feature metadata set corresponding to the current target time is configured as the active user temporal feature metadata set corresponding to the previous time, so as to iteratively update the active user temporal feature metadata set.

[0028] According to a second aspect of the present disclosure, there is provided a data analysis device, comprising:

[0029] An activity user metadata acquisition module is used to acquire original activity data of a target time, and analyze the original activity data to extract an activity user metadata set corresponding to the activity-reached user;

[0030] An active user feature metadata acquisition module, used to extract features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features;

[0031] An active user temporal feature metadata acquisition module, configured to perform feature aggregation based on historical user feature data in combination with the user metadata set and the active user feature metadata set to acquire an active user temporal feature metadata set containing user temporal behavior features;

[0032] The primary user characteristic data acquisition module is used to process the active user temporal characteristic metadata set using preset dimension information to obtain primary user characteristic data containing preset dimensions.

[0033] According to a third aspect of the present disclosure, there is provided a data analysis system, comprising:

[0034] An activity creation module, used to create a current activity task in response to the activity information configured by the user, and register the current activity task with the task management module;

[0035] A task scheduling module, configured to respond to the currently active task, configure task status data corresponding to the currently active task in a relational database, and obtain metadata from an object storage server to execute the above data analysis method;

[0036] The data query module is used to screen target users according to the target features input by the user to obtain secondary target users containing the target features.

[0037] According to a fourth aspect of the present disclosure, there is provided a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned data analysis method is implemented.

[0038] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0039] Processor; and

[0040] A memory, configured to store executable instructions of the processor;

[0041] Wherein, the processor is configured to perform the above-mentioned data analysis method by executing the executable instructions.

[0042] In a data analysis method provided by an embodiment of the present disclosure, the original activity data corresponding to the target time is first obtained, and the data is analyzed for correlation to obtain an activity user metadata set; then the user feature extraction is performed on the activity metadata set to obtain an activity user feature metadata set containing user behavior features; then the data is combined with the historical user feature data, thereby updating the historical user feature data to obtain an activity user temporal feature metadata set containing temporal features corresponding to the target time; by using preset dimension information to analyze and filter the activity user temporal feature metadata, the primary user feature data containing temporal feature changes of each user is obtained, and the user feature changes can be tracked in combination with the temporal features. By obtaining the primary user feature data containing temporal features, it is convenient to mine the target potential users again.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0045] Figure 1 A schematic diagram schematically illustrates a data analysis method in an exemplary embodiment of the present disclosure;

[0046] Figure 2 A schematic diagram schematically illustrates a method for obtaining a set of temporal feature metadata of active users in an exemplary embodiment of the present disclosure;

[0047] Figure 3 A schematic diagram schematically illustrates a data analysis method in an exemplary embodiment of the present disclosure;

[0048] Figure 4 A schematic diagram schematically illustrates a processing method based on 4A state feature data in an exemplary embodiment of the present disclosure;

[0049] Figure 5 A schematic diagram schematically illustrates a processing method based on purchase behavior feature data in an exemplary embodiment of the present disclosure;

[0050] Figure 6 A logical diagram schematically illustrating an iterative calculation in an exemplary embodiment of the present disclosure;

[0051] Figure 7 A schematic diagram schematically illustrates a data analysis device in an exemplary embodiment of the present disclosure;

[0052] Figure 8 A schematic diagram schematically illustrates a data analysis system in an exemplary embodiment of the present disclosure;

[0053] Fig. 9 A schematic diagram schematically illustrates a data architecture in an exemplary embodiment of the present disclosure;

[0054] Fig.10 A schematic diagram schematically illustrates an electronic device in an exemplary embodiment of the present disclosure;

[0055] Fig.11 A schematic diagram schematically illustrates a storage medium in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0057] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0058] This example implementation first provides a data analysis method that can be applied to the evaluation of the effectiveness of a merchant's marketing activities and the re-mining of potential users and high-quality users. Figure 1 As shown in , the above data analysis method may include the following steps:

[0059] S1, obtaining original activity data of a target time, and analyzing the original activity data to extract an activity user metadata set corresponding to the activity-reached users;

[0060] S2, extracting features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features;

[0061] S3, based on the historical user feature data, combining the user metadata set and the active user feature metadata set to perform feature aggregation to obtain an active user temporal feature metadata set containing user temporal behavior features;

[0062] S4, using preset dimension information to process the active user temporal feature metadata set to obtain primary user feature data containing preset dimensions.

[0063] The data analysis method provided in this example implementation can, on the one hand, obtain the primary user feature data of each user including the time series feature changes, and then track the user feature changes in combination with the time series. On the other hand, by obtaining the primary user feature data including the time series features, it is convenient to mine the target potential users again.

[0064] Below, each step of the data analysis method in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0065] In step S1, original activity data of a target time is obtained, and the original activity data is analyzed to extract an activity user metadata set corresponding to the activity-reached users.

[0066] In this example implementation, the above method can be executed collaboratively by the server and the user terminal. Specifically, the user can create an activity task for a specified brand or a specified merchant on the terminal device, and configure specific activity task data; for example, the user can cooperate with activity parameters such as activity type, activity time, channel type, etc. For example, activity types can include advertising, live interaction, coupon delivery, etc.; activity time can be a custom few hours, or multiple consecutive days. After the activity task data is created, it can be stored on the server. Specifically, the server can use OSS data storage service (Object Storage Service). Alternatively, in other exemplary embodiments of the present disclosure, Hive (data warehouse) format can also be used for storage on the server.

[0067] In addition, the above data analysis method can be based on the Spark framework; for the above activity tasks, the task running status table corresponding to the activity tasks can be configured in the MySQL relational database; at the same time, a Spark task can be created to monitor the task running status table, such as an activity task status monitoring task; wherein the task running status can refer to the waiting, running, success and failure status of the Spark task. In addition, for the data results corresponding to the activity tasks, the data calculation status table corresponding to the activity tasks can be configured in the MySQL relational database; at the same time, another Spark task can be created to monitor the data calculation status table, such as a calculation result monitoring task. wherein the data result status can refer to the waiting, production, available, task failure and unavailable status of the operation effect report data.

[0068] In this example implementation, the above step S1 may include:

[0069] Step S11, creating a task to be processed in response to the activity trigger instruction;

[0070] Step S12: Based on the task to be processed, call the target service to obtain the original activity data.

[0071] Specifically, based on the activity tasks created by the user, a pending task corresponding to the activity task can be created based on the activity trigger instruction triggered by the user or based on the activity trigger instruction triggered by the scheduled task. For example, the user can actively trigger the creation of a pending task after obtaining new marketing data; or, a pre-set scheduled task can be used to create a pending task for the updated user data at a specified time every day during the activity.

[0072] When executing the tasks to be processed, the original activity data of the specified time period can be extracted from the OSS server by means of service calls. For example, the original activity data may include the activity type, activity behavior, activity time, etc. for a specified brand or a specified merchant; different types of user activity behavior data tables can also be extracted; for example, the corresponding data tables may include advertising exposure tables, advertising click tables, etc. These tables can be tables in Hive format. After obtaining the above-mentioned original activity data, data cleaning can be performed, and the data tables can be associated with identifiers such as user ID or product ID to extract the activity user metadata set consisting of users reached by the activity. And the activity user metadata set is stored on OSS to complete the data processing of the first interface.

[0073] For example, the data table of the activity-reached users may be shown in Table 1, and the activity user metadata corresponding to the activity-reached users may include the user ID of the activity-reached users and the date information of the corresponding activity-reached users. The activity user metadata set may be represented by Table 1.

[0074] Fields type illustrate dt DATE date user_pin STRING User ID Code

[0075] Table 1

[0076] In step S2, feature extraction is performed on the active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features.

[0077] In this example implementation, after the active user metadata set is collected, a task completion indication message can be generated. Based on the task completion indication message, a processing instruction for the second stage data can be generated. Based on the instruction, the active user metadata set and the user behavior feature table are extracted from the OSS; the user behavior features in the user behavior feature table are associated with the active reached user data in the active user metadata set, which is equivalent to using the user behavior features to perform user screening on the active reached users, thereby obtaining an active user feature metadata set containing user behavior features. Among them, the user behavior features corresponding to the user feature table may include user behaviors such as purchase, browsing, and adding to shopping carts. When performing data association, the user ID or product ID can be used for association. Thus, the data results of the second stage are obtained.

[0078] For example, as shown in Table 2, the active user feature metadata set may include information such as date, user ID, user purchase amount, etc. Among them, user may refer to a user who has made a purchase during the activity, and date may be the purchase time when the activity reaches the user. The active user feature metadata set can be represented by Table 2.

[0079] Fields type illustrate dt DATE date user_pin STRING User ID Code gmv DOUBLE User purchase amount

[0080] Table 2

[0081] In step S3, feature aggregation is performed based on the historical user feature data in combination with the user metadata set and the active user feature metadata set to obtain an active user temporal feature metadata set containing user temporal behavior features.

[0082] In this example implementation, based on the above-mentioned user metadata set of the first stage and the active user feature metadata set of the second stage, the user feature data is combined to aggregate multiple user features. Among them, historical user feature data may include user feature tables such as the 4A status table of existing users, user platform registration schedule, brand fan statistics table, etc. Specifically, these data can be obtained from the OSS server. Among them, "4A" may mean that in order to make marketing activities and user operation strategies more targeted, the 4A user stratification model divides users into four levels of cognition (Aware), attraction (Appeal), action (Act) and advocacy (Advocate), and the relationship is progressive.

[0083] For example, as shown in Table 3, the active user temporal feature metadata set may include multiple fields, including date, user ID, activity reach time series, activity reach user tag time series, activity reach user purchase feature sequence, and purchase information.

[0084]

[0085]

[0086] Table 3

[0087] In this example implementation, specifically, refer to Figure 2 As shown, the above step S3 may include:

[0088] Step S31, classifying users in the user metadata set to obtain first-type users and second-type users corresponding to the current target time;

[0089] Step S32, based on the first type of users corresponding to the target feature and the active user feature metadata set corresponding to the target feature in the previous time, updating the first type of user sequence corresponding to the target feature; and

[0090] Step S33, updating the active user temporal feature metadata set corresponding to the previous time based on the second type of users, to obtain an updated second type user sequence corresponding to the target feature;

[0091] Step S34: constructing the active user temporal feature metadata set corresponding to the target feature within the current target time based on the updated first type user sequence and the second type user sequence.

[0092] In this exemplary implementation, for example, reference Figure 4As shown, taking the user's 4A state sequence as an example, the active user metadata set, the 4A state feature data table, and the 4A state sequence data of the previous day are combined to calculate the current target time, that is, the 4A state sequence data of the latest day.

[0093] After obtaining the above data, you can create a scheduled task for the 4A status feature data, where the scheduled task can be configured with information such as activity time dt and brand code. After entering and executing the scheduled task, you can read the user metadata set to obtain the users reached by the activity channel and read all the users reached by the activity before time dt. And divide the reached users into first-time reach and non-first-time reach. For users who are reached for the first time on dt, combine the 4A status feature data on dt and the day before dt to calculate the sequence [a0, a1] composed of the 4A status on the day before and the day of reach, that is, the first type of user sequence. For users who are not reached for the first time on dt, the 4A status sequence data [a0, a1, ..., an] of each user can be obtained from the time series feature data of the active population on the day before dt. Combined with the 4A status feature data on dt, the 4A status of these users on dt is inserted into the tail of [a0, a1, ..., an] to obtain the 4A status sequence [a0, a1, ..., an, a(n+1)] from the day before the reach to dt, i.e. the second type of user sequence. Then, the updated 4A status sequence data of the first reach and non-first reach users are merged to obtain the 4A status time series calculation result on dt. This can be used to calculate the subsequent population analysis report.

[0094] In this exemplary implementation, for example, reference Figure 5 As shown, the above steps S31 to S34 are explained by taking the purchasing behavior characteristics of users reached by the activity as an example.

[0095] Specifically, the data involved include active user metadata set, purchase feature data, and purchase feature sequence data of the previous day. After creating and inputting a planned task for purchase features, the active reach users shown in Table 1 can be read to obtain all active reach users before the calculation time dt. Combined with the purchase data on the day of dt, the purchase data of the reach users before dt on the day of dt is calculated. According to the user's reach time and purchase time, the valid purchases generated by the activity are selected from all the purchase data. The selection rule is: the purchase occurs within x days after the user reaches. By combining the calculated valid purchase data on the day of dt with the purchase feature data of the day before dt, the users who purchase on the day of dt can be divided into first-time purchasers and non-first-time purchasers, namely, first-type users and second-type users. For users who purchase for the first time on the day of dt, the purchase features can be updated in combination with the historical purchase data, including new and old customer labels in the three dimensions of history, 365 days, and 180 days (i.e., purchase_status_map shown in Table 3). At the same time, the corresponding table of the correspondence between purchase time and purchase amount is updated (i.e., purchase_info_map shown in Table 3). For users who are not first-time buyers on dt, since the purchase features have been updated in a previous iteration, only the corresponding table of purchase time and purchase amount needs to be updated. The updated purchase feature data of first-time buyers and non-first-time buyers are combined to obtain the purchase feature calculation results on dt. This can be used to calculate subsequent purchase analysis reports.

[0096] In this example implementation, the above method may further include: configuring the active user temporal feature metadata set corresponding to the current target time as the active user temporal feature metadata set corresponding to the previous time, so as to iteratively update the active user temporal feature metadata set.

[0097] For example, based on the above method, Figure 6 The iterative calculation strategy shown in the figure can be calculated based on the user feature data table and the activity user metadata set containing the user data that was reached on the first day during the interaction time corresponding to the activity created by the user. On each of the other days, iterative calculations can be performed based on the data of the previous day and the data of the current day, so as to obtain the continuous time series feature change data of the user.

[0098] In step S4, the active user temporal feature metadata set is processed using preset dimension information to obtain primary user feature data containing preset dimensions.

[0099] In this example implementation, after obtaining the active user temporal feature metadata set corresponding to multiple feature categories, data statistics can be performed on the active user temporal feature metadata set according to a preset dimension, thereby obtaining primary user feature data and forming an activity effect analysis data table. The activity effect analysis report data generated based on the primary user feature data can be stored in the ElasticSearch search engine, so that users can read the report data quickly and in real time.

[0100] For example, as shown in Table 4, the primary user feature data may include the cumulative number of new users under each feature tag, the number of user 4A status transfers, the number of new customers under three dimensions, and the number of old customers under three dimensions, etc.

[0101]

[0102] Table 4

[0103] Based on the above, in this exemplary embodiment, reference Figure 3 As shown, the above method may further include:

[0104] Step S5, responding to the target feature input by the user and using the target feature to filter the primary user feature data to obtain secondary target users.

[0105] Specifically, for users, feature keywords can also be used to filter primary user feature data, thereby filtering out potential users with specified features.

[0106] For example, users can pre-configure keywords corresponding to key features, and enter the keywords on the user terminal. The ElasticSearch search engine uses the keywords to filter the target users in the primary user feature data, thereby filtering out the user population containing the target features.

[0107] The data analysis method provided by the present disclosure can obtain the metadata set of active users corresponding to the actual activity reach population through statistics and analysis through the original activity data of multiple channels after the user creates the activity task corresponding to the marketing activity, and obtain the first stage data result. Then, the active user metadata set is associated with the preset user behavior feature table to obtain the active user feature metadata set containing the crowd behavior characteristics, and obtain the second stage data result. Afterwards, based on the first stage data results and the second stage data results, combined with the existing user feature data table, feature aggregation based on time series changes is performed to generate time series feature data of the active population, and a marketing effect analysis report is generated based on the data. By obtaining the time series feature data of the active population, the time series results of the user purchase behavior changes and user tag changes of each user can be accurately obtained, as well as the increase or decrease in the number of each type of users over time. The data changes of each day in the marketing activity can be accurately obtained. Alternatively, in some embodiments, the time period of iterative calculation can also be divided into 1 hour or several hours as needed. The present disclosure can realize the effect analysis method of reaching first and then purchasing and the operation activity effect calculation process based on time series, and can support the analysis of the purchase and other behaviors of the reached people. By selecting the purchase behavior data within x days after reaching, it helps brands evaluate the effective conversion effect generated by the operation activities. In addition, the device can combine the changes in time and use the method of iteratively calculating the user feature time series to obtain the changes in user levels, features and behaviors during the activity cycle, helping brands evaluate the effects of marketing activities from multiple angles. A method for re-identifying key potential groups based on the analysis of operation effects. Based on the user operation effect data, key potential groups can be extracted to help brands reach out again and iteratively improve the effects of marketing activities.

[0108] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0109] For further reference, Figure 7 As shown, in the embodiment of this example, a data analysis device 70 is also provided, including: an active user metadata acquisition module 701, an active user feature metadata acquisition module 702, an active user temporal feature metadata acquisition module 703 and a primary user feature data acquisition module 704, wherein:

[0110] The activity user metadata acquisition module 701 can be used to acquire original activity data of a target time, and analyze the original activity data to extract an activity user metadata set corresponding to the activity-reached users.

[0111] The active user feature metadata acquisition module 702 may be used to extract features of active users in the active user metadata set to acquire an active user feature metadata set containing user behavior features.

[0112] The active user temporal feature metadata acquisition module 703 may be used to perform feature aggregation based on historical user feature data in combination with the user metadata set and the active user feature metadata set to acquire an active user temporal feature metadata set containing user temporal behavior features.

[0113] The primary user feature data acquisition module 704 may be configured to process the active user temporal feature metadata set using preset dimension information to acquire primary user feature data containing preset dimensions.

[0114] In this example implementation, the data analysis device 70 may further include: a first report generating module.

[0115] The first report generating module may be used to generate report data based on the primary user feature data and store the report data in a search engine.

[0116] In this example implementation, the data analysis device 70 may further include: a target user screening module.

[0117] The target user screening module may be used to respond to a target feature input by a user and use the target feature to screen the primary user feature data to obtain secondary target users.

[0118] In this example implementation, the active user metadata acquisition module 701 may also include: a task creation module and a data extraction module.

[0119] The task creation module may be used to create a task to be processed in response to an activity trigger instruction.

[0120] The data extraction module may be used to call a target service based on the task to be processed to obtain the original activity data.

[0121] In this example implementation, the active user feature metadata acquisition module 702 may be used to extract features of active users in the active user metadata set based on preset behavior features, so as to filter users containing the preset behavior features and generate the active user feature metadata set.

[0122] In this example implementation, the active user temporal feature metadata acquisition module 703 may include: a user classification module, a first type user processing module, a second type user processing module and a data integration module. Among them:

[0123] The user classification module may be used to classify users in the user metadata set to obtain first-type users and second-type users corresponding to the current target time;

[0124] The first type user processing module may be used to update the first type user sequence corresponding to the target feature based on the first type user corresponding to the target feature and the active user feature metadata set corresponding to the target feature within a previous time period.

[0125] The second type user processing module may be configured to update the active user temporal feature metadata set corresponding to the previous time based on the second type user, so as to obtain an updated second type user sequence corresponding to the target feature.

[0126] The data integration module may be configured to construct the active user temporal feature metadata set corresponding to the target feature within the current target time based on the updated first type user sequence and the second type user sequence.

[0127] In this example implementation, the active user temporal feature metadata acquisition module 703 may further include: an iteration management module.

[0128] The iteration management module may be used to configure the active user temporal feature metadata set corresponding to the current target time as the active user temporal feature metadata set corresponding to the previous time, so as to iteratively update the active user temporal feature metadata set.

[0129] The specific details of each module in the above-mentioned data analysis device have been described in detail in the corresponding data analysis method, so they will not be repeated here.

[0130] For further reference, Figure 8 As shown, in the implementation of this example, a data analysis system 80 is also provided, including: an activity creation module 801, a task scheduling module 802, and a data query module 803. Among them,

[0131] The activity creation module 801 may be used to create a current activity task in response to activity information configured by a user, and register the current activity task with a task management module.

[0132] The task scheduling module 802 can be used to respond to the currently active task, configure the task status data corresponding to the currently active task in the relational database, and obtain metadata from the object storage server to execute the data analysis method described in the above embodiment.

[0133] The data query module 803 may be used to screen target users according to target features input by the user to obtain secondary target users containing the target features.

[0134] In this example implementation, the above-mentioned data analysis system may further include a data storage module.

[0135] Specifically, the user can create an activity task through the activity creation module 801 and configure the corresponding activity information, wherein the activity information may include parameters such as activity type and activity time, and register the generated activity task to the task scheduling module 802 .

[0136] For the task scheduling module 802, a task status database and a task execution module may be included. The task status database may include a task running status table and a data calculation status table established on a MySQL relational database. The task running status may include the waiting, running, successful and failed states of the Spark task; the data calculation state may refer to the operation effect report data being in the waiting, production, available, task failure and unavailable states. The task execution module may be used to form an execution plan based on the daily data and task status, as well as the activity information registered by the user, and implement data calculation in four stages in order according to the content corresponding to the above data analysis plan.

[0137] In this example implementation, the above-mentioned data analysis system may also include a data storage module. Specifically, this solution involves an active user metadata set for describing user reach time, an active user feature metadata set for describing feature data of an active population, an active user temporal feature metadata set for describing temporal features of an active population, and a report statistics table for describing activity effect analysis. Specifically, the activity effect analysis report statistics table is stored on the ElasticSearch search engine to facilitate fast and real-time reading of report data; the remaining data tables are stored on the OSS server and are read and written through the OSS data storage service.

[0138] In this example implementation, reference Fig. 9As shown, the Spark distributed computing framework can be used, which can include an OSS storage server 92, an ElasticSearch search engine 91, a Spark computing cluster 93, and a Hive data warehouse 94; wherein the Spark computing cluster 93 may include an activity population computing module 931, an activity population feature mining module 932, an activity population time series feature computing module 933, and an effect analysis report computing module 934. For the task execution module, the calculation of user operation effect data can be divided into four stages; wherein the first calculation stage can be to extract the users reached by each activity channel, which is implemented by the activity population computing module. In this stage, relevant activity parameters are obtained from the server, such as the marketing activity behavior table stored in the Hive data warehouse, including activity type, activity behavior, activity time, etc., and different types of user activity behavior hive tables are read, including advertising exposure table, advertising click table, etc. The daily activity reach population data is extracted and stored on the OSS server in the form of activity population metadata. The second calculation stage is to extract the features of the result of the first stage - the active reach population, which is implemented by the active population feature mining module. It reads the metadata of the active reach population in the OSS server, and obtains the user behavior feature hive table from the Hive data warehouse, associates the data to generate the active population features, and stores them on the OSS server in the form of active population feature metadata. The third calculation stage can be based on the calculation results of the first and second stages, combined with the user feature hive tables such as the user 4A status table, user platform registration schedule, and brand fan statistics table in the Hive data warehouse, and implemented by the active population time series feature calculation module, aggregating the user's multiple features, generating active population time series feature data, and storing them on the OSS server in the form of active population time series feature metadata. The fourth calculation stage is based on the results of the third stage, and is implemented by the effect analysis report calculation module, calculating the activity effect data and storing it on the ElasticSearch search engine, which can include a population analysis report containing the characteristics of the population who purchased the product, and a purchase analysis report containing the characteristics of the product purchased, etc. The specific solution details are the same as the embodiment corresponding to the above data analysis method, and will not be repeated here.

[0139] The present invention uses the Spark distributed computing framework to store the activity effect analysis report statistics table on the ElasticSearch search engine, and stores the user contact time data table, activity population feature data table, activity population time series feature combination table, activity effect analysis report statistics table and other data on OSS, and reads and writes through the OSS data storage service; it realizes a user operation effect data evaluation method with scalable computing, storage, and loosely coupled services. In the process of tracking the activity effect, the operation population feature data is mined from the Hive data warehouse, and retained for the user after cleaning and processing by Spark, and stored using OSS, which is convenient for opening up other OSS-related data streams and helping brands to remarket to key feature populations. In addition, the device uses ElasticSearch to store effect analysis data, which is convenient for fast and real-time reading of reports.

[0140] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0141] In an exemplary embodiment of the present disclosure, a computer system capable of implementing the above method is also provided.

[0142] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0143] Refer to the following Fig.10 The computer system 600 of the electronic device according to this embodiment of the present invention is described. Fig.10 The computer system 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0144] like Fig.10 As shown, the computer system 600 is in the form of a general-purpose computing device. The components of the computer system 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).

[0145] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0146] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0147] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0148] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0149] The computer system 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the computer system 600, and / or any device that enables the computer system 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. A display unit 640 may also be connected via the input / output (I / O) interface 650. Furthermore, the computer system 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the computer system 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer system 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0150] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0151] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0152] refer to Fig.11 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0153] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0154] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0155] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0156] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0157] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0158] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0159] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data analysis method, characterized in that: The method comprises: Obtaining original activity data at a target time, and analyzing the original activity data to extract an activity user metadata set corresponding to the activity-reached users; Extracting features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features; Based on historical user feature data, feature aggregation is performed in combination with the user metadata set and the active user feature metadata set to obtain an active user temporal feature metadata set containing user temporal behavior features, including: classifying users in the user metadata set to obtain first-type users and second-type users corresponding to the current target time; updating a first-type user sequence corresponding to the target feature based on the first-type users corresponding to the target feature and the active user feature metadata set corresponding to the target feature in the previous time; and updating the active user temporal feature metadata set corresponding to the previous time based on the second-type users to obtain an updated second-type user sequence corresponding to the target feature; constructing the active user temporal feature metadata set corresponding to the target feature in the current target time based on the updated first-type user sequence and the second-type user sequence; The active user temporal feature metadata set is processed using preset dimension information to obtain primary user feature data containing preset dimensions.

2. The data analysis method according to claim 1, characterized in that: The method further comprises: Report data is generated based on the primary user feature data and stored in a search engine.

3. The data analysis method according to claim 1 or 2, characterized in that: The method further comprises: In response to the target feature input by the user, the primary user feature data is screened using the target feature to obtain the secondary target user.

4. The data analysis method according to claim 1, characterized in that: The obtaining of original activity data comprises: In response to the activity trigger instruction, a pending task is created; Based on the task to be processed, a target service is called to obtain the raw activity data.

5. The data analysis method according to claim 1, characterized in that: The extracting features of the active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features includes: Feature extraction is performed on the active users in the active user metadata set based on preset behavior features to filter users containing the preset behavior features and generate the active user feature metadata set.

6. The data analysis method according to claim 1, characterized in that: The method further comprises: The active user temporal feature metadata set corresponding to the current target time is configured as the active user temporal feature metadata set corresponding to the previous time, so as to iteratively update the active user temporal feature metadata set.

7. A data analysis device, characterized in that: The device comprises: An activity user metadata acquisition module is used to acquire original activity data of a target time, and analyze the original activity data to extract an activity user metadata set corresponding to the activity-reached user; An active user feature metadata acquisition module, used to extract features of active users in the active user metadata set to obtain an active user feature metadata set containing user behavior features; An active user temporal feature metadata acquisition module, configured to perform feature aggregation based on historical user feature data in combination with the user metadata set and the active user feature metadata set to acquire an active user temporal feature metadata set containing user temporal behavior features; A primary user feature data acquisition module, used to process the active user temporal feature metadata set using preset dimension information to acquire primary user feature data containing preset dimensions; The active user temporal feature metadata acquisition module includes: The user classification module is used to classify users in the user metadata set to obtain first-type users and second-type users corresponding to the current target time; The first type user processing module is used to update the first type user sequence corresponding to the target feature based on the first type user corresponding to the target feature and the active user feature metadata set corresponding to the target feature within the previous time period; The second type user processing module is used to update the active user temporal feature metadata set corresponding to the previous time based on the second type user to obtain the updated second type user sequence corresponding to the target feature; The data integration module is used to construct the active user temporal feature metadata set corresponding to the target feature within the current target time based on the updated first type user sequence and the second type user sequence.

8. A data analysis system, characterized in that: include: An activity creation module, used to create a current activity task in response to the activity information configured by the user, and register the current activity task with the task management module; A task scheduling module, configured to respond to the currently active task, configure task status data corresponding to the currently active task in a relational database, and obtain metadata from an object storage server and execute the data analysis method described in any one of claims 1 to 6; The data query module is used to screen target users according to the target features input by the user to obtain secondary target users containing the target features.

9. A storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the data analysis method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the data analysis method according to any one of claims 1 to 6 by executing the executable instructions.

Images