Data Association Analysis Method, System and Storage Medium Based on Member Behavior

Abstract

The present invention belongs to the field of Internet technologies, and discloses a method, a system and a storage medium for data association analysis based on member behavior. The method includes: performing a first scan and a first association parsing on member behavior source data, calculating multiple scenario classification identifiers, and synthesizing an ordered array field of scenario classification identifiers; according to each array field of scenario classification identifiers, summarizing and statistically calculating all dimension items and metric item data under different scenario classifications, and appending them to a full member behavior table; parsing the array field of member behavior through a packaged custom function to parse various behavior data of members under the required scenario and dimension combinations. The method and system disclosed by the present invention save storage resources, reduce calculation overhead, improve data output timeliness, fully improve the usability of data, greatly improve human efficiency, liberate human resources, and at the same time the data has excellent scalability on the premise of ensuring data accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet information processing, and particularly relates to a method and system for data association analysis based on member behavior. Background Art

[0002] Currently, for the association analysis and processing of member behavior in different scenarios, the main process is as Figure 1 shown. It models by dividing tables according to different scenario themes, scans tables multiple times, associates multiple times, and stores multiple copies by scenario. This will cause the phenomena of multiple table reads, multiple repeated associations, and multiple result tables stored in multiple copies partitioned by date, resulting in excessive and repeated use of computing resources, storage resources, and human resources.

[0003] The Internet industry develops rapidly, and enterprise business is increasing day by day. Correspondingly, the dimensions and metrics will increase, and the association logic will also be continuously adjusted and changed. In the prior art, there is a real situation of adjusting the table structure and rebrushing historical data for the result table structure and data, resulting in a large amount of resource calculation and repetitive work.

[0004] Based on the data of multiple result tables, it is often necessary to further complete the analysis of different member categories (such as new and old buyer categories, repeat purchase categories, life cycle categories, high-value RFM categories, etc.), and it is also necessary to calculate and store separately according to their respective algorithms.

[0005] The above-mentioned association analysis and processing method has a large storage capacity for data tables, a large amount of computing resources, and is prone to excessive and repeated use of human resources. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for data association analysis based on member behavior to solve the technical problems in the prior art of large storage capacity of data tables, large computing resources, and easy excessive and repeated use of human resources.

[0007] To solve the above technical problems, in the first aspect of the present invention, a method for data association analysis based on member behavior is provided. The method includes:

[0008] Performing one scan and one association parsing on the member behavior source data to calculate multiple scenario classification identifiers and synthesize an ordered array field of scenario classification identifiers;

[0009] According to each array field of scenario classification identifiers, summarizing and counting all dimension items and metric item data under different scenario classifications and appending them to a full member behavior table;

[0010] Parsing the member behavior array field through a packaged custom function to parse various behavior data of members under the required scenario and dimension combinations.

[0011] Further, when a new analysis scenario is added, the newly added scenario classification identifier is used as a new data element and appended to the scenario classification identifier array field, and a new metric field is extended in the member behavior array field, while other data remains unchanged.

[0012] Further, the member behavior source data includes a main table and a sub-table of member behavior. The sub-table is used for widening or filtering conditions of the main table, including a fact table and other dimension tables. The main table and the sub-table are associated once to obtain different scenario classification identifiers and synthesize an array field of scenario classification identifiers, and then the main table of the source table or the intermediate table is sunk.

[0013] Further, the fields included in the source main table include a source table primary key field, an analysis dimension field, an analysis metric field, and an array field of scenario classification identifiers.

[0014] Further, each scenario contains different dimension combinations. According to the filtering of the scenario classification identifier array, the dimension and index item data under each scenario category are respectively summarized and counted, and the scenario identifier is brought out.

[0015] Further, the full member behavior table includes a scenario identifier, a dimension combination, and a member behavior array. The member behavior array is in the format of time granularity, index item, and value, and is arranged in a certain order.

[0016] Further, various types of behavior data of members obtained through analysis include: new and old buyer types, repurchase rate, life cycle, and high value degree.

[0017] On the other hand, the present invention also provides a data association analysis system based on member behavior, and the system includes:

[0018] A data association module, which is used to perform one scan and one association analysis on the member behavior source data, calculate multiple scenario classification identifiers, and synthesize an ordered array field of scenario classification identifiers;

[0019] A data summarization module, which summarizes and counts all dimension items and index item data under different scenario classifications according to each scenario classification identifier array field, and appends them to a full member behavior table;

[0020] A data analysis module, which parses the member behavior array field through a packaged custom function to parse various types of behavior data of members under the required scenarios and dimension combinations.

[0021] In the third aspect of the present invention, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the method described in the first aspect of the present invention.

[0022] Compared with the prior art, a data association analysis method and system based on member behavior provided by the present invention have the following technical effects:

[0023] 1. For the source table data of member behavior in the present invention, it only needs to be calculated and summarized by means of one-time reading, one-time association, and adding an array field of scenario classification identification to the main table redundantly. The operation is simple and efficient, and it greatly saves storage resources and computing resources.

[0024] 2. Through the association method of the present invention, when the demand association logic needs to be adjusted urgently, the original logic can be replaced quickly and efficiently, and at the same time, the expansion of downstream application scenarios can be supported; the initialization of historical data can also be associated through a single traversal of the entire table, and the full volume can be updated.

[0025] 3. Based on the full-scale member behavior table and its array field of full-scale member behavior in the present invention, by encapsulating UDF functions (user-defined functions), member behavior label identities such as new and old buyer classes, repeat purchase classes, life cycle classes, high-value RFM classes, etc. are parsed and extracted, which flexibly and efficiently supports member analysis under different scenario and dimension combination granularities. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for associative analysis of member behavior data in the prior art.

[0027] Figure 2 is a flowchart of the data association analysis method based on member behavior according to an embodiment of the present invention.

[0028] Figure 3 is a comparison chart of the processing flows of the prior art processing solution and the processing solution of the present invention.

[0029] Figure 4 is an architecture diagram of the data association analysis system based on member behavior according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] The following embodiments are only used to illustrate the technical solutions of the present invention more clearly, and should not be used to limit the protection scope of the present invention. For example, certain terms are used to refer to specific components in the specification and claims. Those skilled in the art should understand that hardware or software manufacturers may use different terms to refer to the same component. The specification and claims do not use the difference in names as a way to distinguish components, but use the difference in functions of components as a criterion for distinction. The subsequent description in the specification is the preferred implementation mode for implementing the present invention, but the description is for the purpose of explaining the general principles of the present invention, not for limiting the scope of the present invention. The protection scope of the present invention shall be subject to what is defined by the appended claims.

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] As Figure 2 , Figure 3 shown, an embodiment of the present invention discloses a data association analysis method based on member behavior, including the following steps:

[0033] Step 1: Perform a first scan and a first association analysis on the source data of member behavior, calculate multiple scenario classification identifiers, and synthesize an ordered array field of scenario classification identifiers.

[0034] By reading the data source table involved in member behavior at one time, associating the source tables for data, parsing out the scenario classification identifiers and synthesizing an ordered array field of scenario classification identifiers, and storing them in the main table of the source table, the primary key field (or combined primary key field) of the source table, the analysis dimension field (including the member code field), the analysis metric field, and the array field of scenario classification identifiers can be obtained.

[0035] Member behavior mainly generates behavior data when members place orders. Taking the order behavior of members as an example, through processing the order behavior data, member class analysis results under different scenario and dimension combinations are obtained. For example, from three scenario perspectives (main station operation perspective, brand supplier perspective, store perspective), member behavior data such as new and old buyers, repurchase rate, lifecycle, high-value, etc. for different product categories are viewed. Among them, the source data of order behavior is mainly the main table represented by the daily table recording member order behavior, as well as other slave tables, including relevant fact tables and relevant dimension tables. The slave tables are mainly used for widening the main table and can also be used as filtering conditions during the screening process. Read the data source table once, associate the main table and the slave tables at one time, merge the three scenario classification identifiers into an array field of classification identifiers, and sink them into the main table of the source table or an intermediate table. Each scenario contains different dimension combinations. According to the screening of the array field of classification identifiers, the dimension items and all index items under each scenario category are respectively summarized and counted, and the scenario identifier is brought out.

[0036] Step 2: According to each array field of scenario classification identifiers, summarize and count all dimension items and index item data under different scenario classifications, and append them to a full member behavior table.

[0037] Each scenario contains different dimension combinations. According to the screening of the array field of scenario classification identifiers, the dimension combinations and all index items to be analyzed under each scenario are respectively summarized and counted. According to the array field of scenario classification identifiers and the dimension combination field, the scenario identifier is brought out respectively.

[0038] As shown in Table 1 below, all dimension and all indicator item data are appended to a full member behavior scale table to obtain a data structure including a scenario identifier, all analysis dimensions, and a full member behavior array. Among them, the structure representation of the full member behavior array can be "{time granularity 1: {indicator item 1: value 1;...; indicator item n: value n};...; time granularity m: {indicator item 1: value 1;...; indicator item n: value n}}".

[0039] Table 1 Data Structure of the Full Scale Table

[0040]

[0041] An exemplary full record taking the order behavior data of members as an example is shown in Table 2 below:

[0042] Table 2 Sample Table of Member Order Behavior Data

[0043]

[0044]

[0045] In this step, different dimension combination examples can be formed. Since a full behavior array with a scenario identifier is formed, the data in the member behavior source table only needs to be calculated and summarized by the method of reading once, associating once, and adding a scenario classification identifier array field redundantly to the main table, which is simple, efficient, and saves storage resources and computing resources.

[0046] Step 3: Parse the member behavior array field through the encapsulated custom function to parse out various behavior data of members under the required scenarios and dimension combinations.

[0047] The full member behavior array contains the full behavior data of each member. By encapsulating the UDF function to parse the member full behavior array field, such as parsing the number of repeat-purchasing members of all stores within any time period (the UDF function can obtain the purchase situation during this period by passing in the start time and end time, judge whether there is a repeat purchase, and then calculate the number of repeat-purchasing members); similarly, it can support most member analysis scenarios, such as new and old buyers, repeat purchase retention, life cycle, FRM and other member class indicators on the dimension combination under a detailed scenario. For example, to count the sub-new buyers of a certain store in the current year from the store perspective, with the scenario identifier limited to a scenario, the overall first purchase date of the members can be obtained through the window function. By encapsulating the function to parse the member behavior array, the store first purchase date of all members in all stores can be obtained. When the store first purchase date occurs in the current year and is less than the overall first purchase date, it is a sub-new buyer. By removing duplicates and summarizing, the number of sub-new buyers, the order volume of sub-new buyers, the sales quantity of sub-new buyers, the payment amount of sub-new buyers and other related data can be counted, and these data can be used for the analysis of staff.

[0048] In this step, based on the full member behavior scale and the behavior array fields within its full member scale, by encapsulating UDF functions, member behavior label identities such as new and old buyer categories, repurchase categories, life cycle categories, high-value RFM categories, etc. are parsed and extracted, flexibly and efficiently supporting member analysis under different scenarios and dimensional combination granularities.

[0049] In addition, in a preferred embodiment of the present invention, when a new analysis scenario is added, a new scenario classification identifier is added as a new array element on the basis of step 1 and appended to the array field, and the length of the array field is increased accordingly; new metric fields or dimensional fields are extended on the full member behavior scale, and the existing data remains unchanged; then, after being processed by step 3, the new scenario identifier, all analysis dimensions, and the full member behavior array data are obtained and appended to the full scale table. The above adjustments enable the rapid and efficient replacement of the original logic when the demand association logic urgently needs to be adjusted, while supporting the expansion of downstream application scenarios; the initialization of historical data can also be associated once by traversing the entire table once, and the full volume is updated.

[0050] The data association method in the embodiments of the present invention reads, associates, stores, and retrieves the data of the member behavior source table once, effectively avoiding data recalculation, greatly saving the storage resources and computational resource overhead; stores in full volume according to the scenario + dimension combination granularity, and all member behavior information is stored in an array field. Various member class analysis scenarios can call the UDF function to parse this array field, which is fast and flexible, and can be used in multiple ways with one table, saving the resource consumption in the process of separately establishing each label; greatly improving the human efficiency, saving storage costs and computational overhead, and flexibly supporting the analysis of a series of member behaviors such as new and old member categories, repurchase and retention categories, life cycle categories, high-value RFM categories, etc. under different dimensional combination scenarios. On the premise of ensuring data accuracy, it saves storage resources, reduces computational overhead, improves the timeliness of data output, fully improves the usability of data, greatly improves human efficiency, liberates manpower, and at the same time the method itself has excellent scalability.

[0051] In addition, alternatively, the above method can be implemented by a computer program product, i.e., a computer-readable storage medium. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present invention. The computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the above. The computer-readable storage medium used herein is not construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0052] Please refer again to Figure 4 As shown, another embodiment of the present invention further provides a data association analysis system based on member behavior. The system includes: a data association module, a data summary module, and a data analysis module. Among them,

[0053] The data association module is used to perform a first scan and a first association analysis on the member behavior source data, calculate multiple scenario classification identifiers, and synthesize an ordered array field of scenario classification identifiers;

[0054] The data summary module, according to each array field of scenario classification identifiers, summarizes and statistically calculates all dimension items and metric item data under different scenario classifications, and appends them to a full member behavior table;

[0055] The data analysis module parses the member behavior array field through an encapsulated custom function to obtain various behavior data of members under the required scenario and dimension combinations.

[0056] In addition, the data association analysis system in this embodiment further includes an extension module for adding a new array of scenario classification identifiers to the array field and updating the full member behavior table.

[0057] Through the data association analysis system in the embodiments of the present invention, it is possible to achieve a single scan and association of the member behavior source table, which is applicable to multi-scenario call analysis, effectively saving computing power by up to 70% and storage by more than 80%. It can support frequent adjustment and change of association logic without modifying the table structure and increasing the development workload, making it easier to implement. Based on the member full-behavior full-scale table, by parsing the array fields, it is also possible to efficiently and flexibly support member scenario analysis under different dimension combinations, fully improving the usability of the data, greatly improving the human efficiency, liberating the manpower, and also having good scalability.

[0058] It should be noted that the above are only the preferred embodiments of the present invention, and thus do not limit the patent protection scope of the present invention. The present invention can also improve the materials and structures of the above various components, or use technical equivalents for replacement. Therefore, any equivalent structural changes made by using the specifications and illustrations of the present invention, or directly or indirectly applied to other related technical fields, are similarly included in the scope covered by the present invention.

Claims

1. A method for data association analysis based on member behavior, characterized in that The method includes: Performing a single scan and a single associated parsing on the member behavior source data, calculating multiple scenario classification identifiers, and synthesizing an ordered array field of scenario classification identifiers. Each scenario contains different dimension combinations. According to the filtering of the scenario classification identifier array, the dimension items and metric items data under each scenario category are respectively summarized and counted, and the scenario identifier is carried out; According to each scenario classification identifier array field, the data of all dimension items and metric items under different scenario classifications are summarized and counted, and appended to a full member behavior table to obtain a data structure including the scenario identifier, all analysis dimensions, and the full member behavior array, forming a full member behavior array with scenario identifiers; Parsing the member behavior array field through an encapsulated custom function to obtain various behavior data of members under the required scenarios and dimension combinations. Among them, the full member behavior array field is parsed by encapsulating a UDF function.

2. The data association analysis method based on member behavior according to claim 1, wherein When a new analysis scenario is added, the newly added scenario classification identifier is used as a new data element and appended to the scenario classification identifier array field, and a new metric field is extended in the member behavior array field, while other data remains unchanged.

3. The method for data association analysis based on member behavior according to claim 1 or 2, characterized in that, The member behavior source data includes a member behavior main table and a sub-table. The sub-table is used for widening or filtering conditions of the main table, including a fact table and other dimension tables. The main table and the sub-table are associated once to obtain different scenario classification identifiers and synthesize a scenario classification identifier array field, and then the main table of the source table or the intermediate table is sunk.

4. The data association analysis method based on member behavior according to claim 3, wherein The fields included in the main table of the source table include the source table primary key field, the analysis dimension field, the analysis metric field, and the scenario classification identifier array field.

5. The data association analysis method based on member behavior according to claim 1, wherein The full member behavior table includes a scenario identifier, a dimension combination, and a member behavior array. The format of the member behavior array is the time granularity, metric item, and value set in sequence.

6. The data association analysis method based on member behavior according to claim 1, wherein, The various behavior data of the members obtained through analysis include: new and old buyer types, repurchase rate, life cycle, and high value degree.

7. A data association analysis system based on member behavior, characterized in that, The system includes: A data association module for performing a single scan and a single associated parsing on the member behavior source data, calculating multiple scenario classification identifiers, and synthesizing an ordered array field of scenario classification identifiers. Each scenario contains different dimension combinations. According to the filtering of the scenario classification identifier array, the dimension items and metric items data under each scenario category are respectively summarized and counted, and the scenario identifier is carried out; A data summarization module that summarizes and counts the data of all dimension items and metric items under different scenario classifications according to each scenario classification identifier array field, and appends them to a full member behavior table to obtain a data structure including the scenario identifier, all analysis dimensions, and the full member behavior array, forming a full member behavior array with scenario identifiers; A data analysis module that parses the member behavior array field through an encapsulated custom function to obtain various behavior data of members under the required scenarios and dimension combinations. Among them, the full member behavior array field is parsed by encapsulating a UDF function.

8. The data association analysis system based on member behavior according to claim 7, wherein The system further includes an extension module for adding a new scenario classification identifier array to the original array field and updating the full member behavior table.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the method according to any one of claims 1 to 6 above.

Images