A data processing method, device, server equipment and chip system

Abstract

The embodiments of the present application provide a data processing method, apparatus, server device and chip system, which relate to the field of data processing. The method can effectively improve the computing efficiency of the existing population. The method may include: the first processing module sends a first SQL message to the computing module, and the first SQL message is used to indicate all the label information in the first condition group, and the logical connection relationship between the various label information configured in the first condition group. The first condition group is included in the at least one condition group. The computing module determines the first population bitmap based on the first comparison table and the first SQL information. The first population bitmap corresponds to the first condition group. Among them, the first comparison table includes the population bitmap corresponding to each label information in the first condition group.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular, to a data processing method, apparatus, server device, and chip system. Background Art

[0002] Currently, the server configured in the cloud can classify users into different population sets based on the behavioral data of a large number of users and the configured condition groups. In this way, the users in each population set can meet the labels of the corresponding condition groups.

[0003] In order to ensure the accuracy of the crowd set, the server can continuously update the user's behavior data, and according to the configured condition group, according to the updated behavior data, continuously update the crowd set corresponding to each condition group. In the existing implementation scheme, the update processing method for the crowd set will occupy a lot of computing power and time resources of the server, and the computing efficiency is low. Summary of the invention

[0004] The present application provides a data processing method, device, server equipment and chip system, which can effectively improve the computing efficiency of the existing population.

[0005] In order to achieve the above technical objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a data processing method is provided, which is applied to a data processing device, and the data processing device is configured with a first processing module and a computing module. The first processing module is configured with at least one condition group, each of which includes at least one tag information, the tag information includes a tag, and the tag is configured with a tag value. The method includes: the first processing module sends a first SQL message to the computing module, and the first SQL message is used to indicate all the tag information in the first condition group and the logical connection relationship between the various tag information configured in the first condition group. The first condition group is included in the at least one condition group. The computing module determines a first population bitmap based on a first comparison table and the first SQL information. The first population bitmap corresponds to the first condition group. The first comparison table includes the population bitmap corresponding to each tag information in the first condition group.

[0007] Based on this solution, the first processing device can instruct the calculation module to determine the bitmaps corresponding to the various labels that need to be calculated based on the prepared population bitmaps corresponding to the various label information and the labels of the first condition group to be calculated. Then, according to the logical connection relationship between each bitmap and each label, a simple and / or calculation can be performed to quickly obtain the population bitmap of the first condition group. Among them, the first condition group can correspond to the existing population.

[0008] Optionally, after determining the first group information, the method further includes: the calculation module storing a correspondence between the first group identifier and the first group bitmap. The first group identifier corresponds to the first condition group.

[0009] Optionally, the data processing device is further configured with a second processing module and a storage module, and the method further includes: the first processing module controls the second processing module to obtain the corresponding relationship between the first population identifier and the first population bitmap. The first processing module controls the second processing module to obtain the first user identifier set according to the first population bitmap. The first processing module controls the second processing module to send the first user identifier set and the first population identifier to the storage module.

[0010] Optionally, the method further includes: the first processing module controlling the storage module to store the correspondence between the first group identifier and the first user identifier set.

[0011] Therefore, in addition to storing the bitmap in the calculation module, the above solution can also be used to store multiple user identifiers corresponding to the first condition group that can be directly used in the storage module.

[0012] Optionally, before the first processing module sends the first SQL information to the calculation module, the method further includes: the calculation module obtains the first comparison table under the control of the first processing module.

[0013] Optionally, the calculation module obtains the first comparison table under the control of the first processing module, including: the first processing module controls the calculation module to obtain the first behavior data table. The first behavior data table includes at least one behavior data, and the behavior data in the first behavior data table corresponds to the behavior data in the second behavior data table. The second behavior data table is a behavior data table configured in the storage module. The second behavior data table includes at least one behavior data, and the behavior data includes a user identifier and at least one tag information corresponding to the user identifier. The first processing module controls the calculation module to obtain the first comparison table according to the first behavior data table. Thus, a mechanism for obtaining the first comparison table is disclosed.

[0014] Optionally, the first processing module controls the computing module to obtain the first behavior data table, including: after the first processing module completes updating the second behavior data table, the first processing module sends a first transmission instruction to the second processing module. In response to the first transmission instruction, the second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the computing module.

[0015] Two possible behavior data synchronization methods are provided below.

[0016] Optionally, the second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the calculation module, including: the second processing module obtains the second behavior data from the second behavior data table, the second behavior data includes first label information, and the type of the first label information is a nested mix type. The second processing module splits the first label information to obtain the first sub-data and the second sub-data, the first sub-data and the second sub-data respectively represent label information of different dimensions in the first label information, and the first sub-data and the second sub-data are both label information of a single dimension. The second processing module stores the first sub-data and the second user identifier as the first table item of the first behavior data table. The second processing module stores the second sub-data and the second user identifier as the second table item of the first behavior data table. The second user identifier is the user identifier corresponding to the second behavior data.

[0017] Optionally, the mix type label information includes label information of at least two dimensions.

[0018] Optionally, the second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the calculation module, including: the second processing module obtains third behavior data from the second behavior data table, the third behavior data includes second label information, and the type of the second label information is non-mix type. The second processing module stores the correspondence between the second label information and the third user identifier in the third table item of the first behavior data. The third user identifier is the user identifier corresponding to the third behavior data.

[0019] Therefore, the behavior data synchronized to the calculation module does not include the label information of the mix type, thereby improving the calculation efficiency in the calculation module.

[0020] Optionally, the non-mix type label information includes: dimension array type label information, and / or other types of label information. Each non-mix type of label information includes label information of one dimension. For example, the other types of label information may include enumeration type, numeric type, date type, and other label information.

[0021] Optionally, the first processing module controls the calculation module to obtain the first comparison table according to the first behavior data table, including: the first processing module generates a second SQL message, and the second SQL message includes all configured label information. The first processing module sends the second SQL message to the calculation module. The calculation module determines the population bitmap corresponding to each label information indicated by the second SQL message based on the second SQL message and the first behavior data table. The calculation module stores the correspondence between each label information indicated by the second SQL message and the corresponding population bitmap in the first comparison table. Therefore, through the second SQL message, the first processing module can control the calculation module to obtain the bitmap corresponding to each label information that may be used before calculating the existing population.

[0022] Optionally, a second comparison table is stored in the first processing module, and the second comparison table stores a correspondence between at least one population identifier and metadata. Among them, the correspondence between at least one population identifier and metadata includes: a correspondence between the first population identifier and the first metadata. The first metadata includes label information indicated by the first condition group. The first processing module generates the second SQL information, including: the first processing module generates the second SQL information according to the second comparison table. Thus, a solution for generating the second SQL information is provided. In this example, the first processing module may store metadata corresponding to the condition group of all existing populations. In this way, the first processing module can determine all label information that may be used based on the metadata corresponding to all existing populations, and then generate the second SQL information accordingly.

[0023] Optionally, the method also includes: the first processing module sends a third SQL message to the calculation module, and the third SQL message is used to indicate all the label information in the second condition group, and the logical connection relationship between the various label information configured in the second condition group. The second condition group is obtained after the first processing module rearranges the various labels in the first condition group. The rearrangement process includes: for each label group in the first condition group, adjusting one or more labels of the mix type to the head of the label group, and adjusting one or more labels of the array type or other types to the tail of the label group. The calculation module determines the second population bitmap corresponding to the first condition group based on the third SQL message and the second behavior data table before the update stored in the storage module.

[0024] This solution provides an efficient way to calculate new populations. Based on the calculation scheme for the existing population, for the calculation of new populations, the first processing module can rearrange the configured condition groups to obtain the separate aggregation processing forms of mix types and other types of labels that the calculation module can process more efficiently. Therefore, according to the second condition group obtained after the rearrangement, SQL information is generated, and the first processing module can control the calculation module to calculate the new population more efficiently.

[0025] Optionally, the first processing module may be a DMP server or a DMP processing platform. The second processor may be a SPARK engine or a SPARK server. The storage module may be a HIVE server. The computing module may be a ClickHouse computing platform or server.

[0026] In a second aspect, a data processing device is provided, the data processing device being configured with a first processing module, a second processing module, a storage module, and a computing module. The data processing device is used to determine first population information corresponding to a first condition group according to the method provided in the first aspect and any possible implementation thereof. The first population information includes a first population bitmap, and / or the first population information includes a first user identification set.

[0027] In a third aspect, a server device is provided, wherein the server device is configured with a first processing module, and when the server device is in operation, the server device is used to implement the function of the first processing module in the first aspect and any possible design thereof. For example, the server device may be a DMP server.

[0028] In a fourth aspect, a chip system is provided, which is configured in a server device as provided in the third aspect. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line. The interface circuit is used to receive a signal from the memory of the electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory. When the processor executes the computer instruction, the server device is used for the function of the first processing module in the first aspect and any possible design thereof.

[0029] In a fifth aspect, the present application also provides a chip system, which is applied to a server device as provided in the third aspect, or in a first processing module as provided in the first aspect. The chip system may include one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line, and the interface circuit is used to receive a signal from the memory and send the signal to the processor, where the signal includes a computer instruction stored in the memory. When the processor executes the above-mentioned computer instruction, the server device or the first processing module is used to implement the relevant functions provided in the above-mentioned first aspect and any possible implementation thereof.

[0030] In a sixth aspect, the present application also provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enable a server device or a first processing module to implement the relevant functions provided in the above-mentioned first aspect and any possible implementation thereof.

[0031] In a seventh aspect, the present application also provides a computer program product, which, when executed on a computer, enables the computer to implement the relevant functions provided in the above-mentioned first aspect and any possible implementation thereof.

[0032] It can be understood that the solutions provided in the second aspect to the seventh aspect of the present application can respectively correspond to the first aspect and any possible design thereof, so the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a communication scenario;

[0034] Figure 2 A logical diagram for obtaining crowd information for a cloud-based process;

[0035] Figure 3 It is a schematic diagram of the logical composition of a condition group;

[0036] Figure 4 It is a schematic diagram of obtaining crowd information according to condition groups;

[0037] Figure 5 A schematic diagram of the composition of a data processing device provided in an embodiment of the present application;

[0038] Figure 6 A logic diagram of a label type provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the logical composition of a condition group provided in an embodiment of the present application;

[0040] Figure 8A flowchart of a data processing method provided in an embodiment of the present application;

[0041] Fig. 9 A schematic diagram of a rearrangement process provided in an embodiment of the present application;

[0042] Fig.10 A flowchart of a data processing method provided in an embodiment of the present application;

[0043] Fig.11 A flowchart of a data processing method provided in an embodiment of the present application;

[0044] Fig.12 A schematic diagram of the composition of a server device provided in an embodiment of the present application;

[0045] Fig.13 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0047] At present, in the process of providing services to users, electronic devices can interact with the cloud connected to the communication, so that the electronic devices can obtain corresponding information through the cloud. The cloud may include a server, or the cloud may include multiple servers working together.

[0048] Exemplary, reference Figure 1 , which is a schematic diagram of a communication scenario.

[0049] In such Figure 1 In the example, the electronic devices connected to the cloud for communication may include electronic device U1, electronic device U2, and electronic device U3.

[0050] For the cloud, users can be divided into different groups based on their behavior data of using various electronic devices and multiple rule conditions configured in the cloud.

[0051] In the present application, a group of the multiple rule conditions can be referred to as a condition group. Each condition group can include multiple tag information. Each tag information can correspond to a judgment logic. In this way, multiple tag information are logically connected (such as and, or, etc. logical connection forms) to form a complete condition group. In some implementations, the tag information may include a tag. Some tags may also be configured with a tag value. In the following description, a tag also represents tag information. Unless otherwise specified, a tag may include a corresponding configured tag value.

[0052] In an embodiment of the present application, the behavior data may include a user identifier (such as the SN of a device used by the user) and corresponding one or more tag information.

[0053] Thus, the cloud can classify one or more users in the multiple behavior data that meet the labels of the corresponding condition group and the logical connection relationship between the labels into a population set. The population set can be the population set corresponding to the condition group.

[0054] As an example, refer to Figure 2 , the cloud can be configured with condition group CG1 and condition group CG2. Figure 3 As shown, the condition group CG1 can be configured with a tag S11 and a tag S12. The tag S11 and the tag S12 can be logically connected to communicate and constitute the condition group CG1. In addition, the condition group CG2 can be configured with a tag S21 and a tag S22. The tag S21 and the tag S22 can be logically connected to communicate and constitute the condition group CG2.

[0055] The cloud can classify the users corresponding to each electronic device into the population set corresponding to condition group CG1 or condition group CG2 based on the behavior data (such as behavior data 11, behavior data 12, and behavior data 13, etc.) directly or indirectly obtained from the electronic device.

[0056] In this example, the crowd set corresponding to the condition group CG1 can be identified by crowd information Cr1. The crowd information Cr1 can include the corresponding identification of one or more users who meet the condition of the condition group CG1. The corresponding identification of the user can be a device ID, a user ID, a device serial number (Serial Number, SN), etc.

[0057] For example, Figure 1 The following scenario is used as an example. Figure 4 , the cloud can classify the users corresponding to the electronic device U1 and the electronic device U2 into corresponding crowd sets according to the condition group CG1. Take the crowd information corresponding to the crowd set including the SN of each user's device as an example. Figure 4As shown, the crowd information Cr1 may include SN1 corresponding to the electronic device U1, and SN2 corresponding to the electronic device U2.

[0058] Similarly, the crowd set corresponding to CG2 configured in the cloud can be identified by crowd information Cr2. For example, the cloud can classify the electronic device U3 into the corresponding crowd set according to the condition group CG2. For example, the crowd information Cr2 corresponding to the condition group CG2 can include SN3 corresponding to the electronic device U3.

[0059] Therefore, the cloud can obtain the crowd information corresponding to each condition group according to the behavior data and the configured condition group. Based on this, the cloud can provide corresponding customized services for each crowd information.

[0060] Above Figures 1 to 4 The examples are only examples. In actual implementation, the number of electronic devices connected to the cloud is much greater than Figure 1 For example, the number of electronic devices can reach hundreds of millions. The number of tags that make up the conditional group configured in the cloud is also much larger than the above example. For example, the number of currently effective tags may reach 600 or more.

[0061] In this way, the cloud can Figures 1 to 4 A similar implementation is shown, where the large number of users are divided into the population information corresponding to each condition group.

[0062] Generally, the behavior data obtained by the cloud can be continuously updated. For example, the cloud can obtain and update the behavior data once a day. In this way, for each configured condition group (such as condition group CG1 and condition group CG2), the cloud needs to update the crowd information corresponding to each condition group every day based on the updated behavior data.

[0063] For example, when the cloud updates the crowd information corresponding to each condition group, each updated behavior data can be judged according to each condition group to determine whether the behavior data meets the conditions corresponding to the condition group. The cloud can thus determine whether the user corresponding to each behavior data is still included in the crowd set corresponding to the current condition group.

[0064] In the embodiment of the present application, after the configuration of the condition group is completed in the cloud, the process of determining the crowd information corresponding to the condition group for the first time can be called the calculation process of the new crowd. Correspondingly, after the calculation process of the new crowd is completed, the process of the cloud continuously calculating the crowd information of each condition group based on the updated behavior data can be called the calculation process of the existing crowd.

[0065] In actual implementation, the calculation of the existing population will occupy a large amount of computing resources. For example, the calculation process of the existing population may occupy 90% or even higher of the entire computing resources of the cloud. The calculation time of the existing population is also high, such as up to 5 hours or more.

[0066] Correspondingly, the technical solution provided in the embodiment of the present application can make the cloud more efficient in calculating the existing population. In some embodiments, based on the technical solution provided in the embodiment of the present application, the calculation process of the newly created population can also be optimized, thereby reducing the calculation overhead of the newly created population and improving the calculation efficiency of the newly created population.

[0067] refer to Figure 5 , is a schematic diagram of the composition of a data processing device provided in an embodiment of the present application. The technical solutions provided in each embodiment of the present application can be implemented by the following example: Figure 5 The composition shown is implemented.

[0068] like Figure 5 As shown, in this example, the data processing device may include a first processing module, a second processing module, a storage module, and a computing module.

[0069] The storage module can be used to store massive amounts of data. In some embodiments, the storage module can be implemented by HIVE. HIVE is a data warehouse tool based on Hadoop. HIVE can be used to extract, transform, and load data.

[0070] In some embodiments of the present application, the storage module can be used to store massive amounts of behavior data. Among them, the processing mechanism for HIVE to obtain concurrent behavior data can be executed after each user is authorized by the electronic device.

[0071] In some other embodiments of the present application, the storage module can also be used to store crowd information corresponding to each condition group. Wherein, the crowd information can include one or more user identifiers. In combination with the above description, in some implementations, the user identifier can be the SN of the electronic device used by the user. In this implementation, the storage module can also store the crowd identifiers corresponding to each crowd. In this way, when it is necessary to obtain the crowd corresponding to a certain condition group, the corresponding crowd information can be obtained from the storage module according to the crowd identifier corresponding to the condition group.

[0072] The first processing module is used to control the work of each module component. In some embodiments, the first processing module can realize corresponding functions through a data management platform (Data Management Platform, DMP).

[0073] As a possible implementation, the first processing module can be used to control other components to cooperate in calculating the new population when the behavior data is updated and the new condition group is configured.

[0074] As another possible implementation, the first processing module can control other components to cooperate in calculating the existing population when the behavior data is updated.

[0075] In some embodiments, the first processing module can also be used to provide condition group configuration capabilities. For example, the first processing module can provide a UI interface for configuring condition groups to developers through a connected display screen. In this way, the first processing module can generate and store the configured condition groups based on the operations received on the UI interface.

[0076] The calculation module is used to determine the population information corresponding to each condition group under the control of the first processing module. In some implementations, the calculation module can be implemented by ClickHouse. ClickHouse is a column-based database management system (DBMS) for online analytical processing (OLAP).

[0077] Exemplarily, the computing module may be used to obtain control information carrying a structured query language (SQL) from the first processing module. The control information carrying the SQL field may correspond to the judgment logic of the condition group. That is, the SQL field of the control information may correspond to the judgment logic represented by each tag in the indicating condition group, and the logical relationship between each tag. In the following description, the SQL field is referred to as SQL.

[0078] The calculation module can also obtain current behavior data from the storage module. The current behavior data can be the behavior data obtained before the calculation of the new population. Alternatively, the current behavior data can be the updated behavior data stored in HIVE before the calculation of the existing population.

[0079] The calculation module can also determine the population information corresponding to the condition group corresponding to the current SQL according to the SQL indicated by the control information and the behavior data.

[0080] like Figure 5 As shown, in the present application, the data processing device may also be configured with a second processing module. The second processing module may be used to implement the transmission of data from the storage module to the computing module.

[0081] For example, the second processing module can obtain behavior data from the storage module under the control of the first processing module. The second processing module can convert the data in the user's behavior data that conforms to a preconfigured type (such as a mix type). The mix type data is thereby converted into two or more array type data or other types of data of equal length. The second processing module can also transfer the converted data to the computing module. In addition, for array type data or other types of data, the second processing module can directly transfer the behavior data to the computing module under the control of the first processing module.

[0082] In the embodiment of the present application, the user's behavior data may include multiple different types, such as mix type, array type, and other types. The description of the different types will be described in detail later.

[0083] Therefore, based on Figure 5 The data processing device shown can realize efficient calculation of new and existing populations in the cloud through the solution provided by the embodiment of the present application.

[0084] It should be noted that the cloud involved in the embodiments of the present application may include one or more servers.

[0085] Corresponding to Figure 5 The data processing device shown can, in some implementations, be deployed in the same server in the cloud.

[0086] In some other implementations, different module components in the data processing device can also be deployed in different servers in the cloud. For example, the first processing module can be configured on the DMP platform server, the second processing module can be configured on the spark server, the storage module can be configured on the HIVE server, the computing module can be configured on the ClickHouse server, etc.

[0087] In this way, the servers are connected in communication and work together to implement the functions of the data processing device in the embodiment of the present application.

[0088] In the following description, Figure 5 Taking the composition shown as an example, the implementation of the solution provided in the embodiment of the present application is explained.

[0089] In order to more clearly illustrate the implementation of the solution provided by the embodiments of the present application, the following first briefly describes the behavior data, condition groups, and label types involved in the embodiments of the present application.

[0090] Combination Figure 5As described in , no matter whether the calculation of a new population or the calculation of an existing population is performed, multiple behavior data can be stored in HIVE. The multiple behavior data can be used as the basis for calculating a new population or an existing population.

[0091] In the present application, each behavior data may correspond to a user identifier (such as a SN) and a label value corresponding to multiple labels.

[0092] As an example, the following Table 1 provides an example of behavior data. The behavior data in Table 1 may correspond to one of a plurality of behavior data stored in HIVE.

[0093] Table 1

[0094] User ID Installed APP gender age …… 0001 ["Application 1","Application 2"] male 25 ……

[0095] As shown in Table 1, the user ID corresponding to the behavior data may be 0001. In some implementations, 0001 may be replaced with the SN of the device currently used by the user. In the example of Table 1, the tags included in the current behavior data may include: "installed APP", "gender", "age", etc.

[0096] In this way, through this behavioral data, it can be represented that for a user with ID 0001, in the "Installed APP" label, the corresponding label value is "["Application 1", "Application 2"]"; in the "Gender" label, the corresponding label value is "Male"; in the "Age" label, the corresponding label value is "25", and so on.

[0097] It should be noted that the behavior data shown in Table 1 is only an example and does not constitute a limitation on the behavior data in the embodiments of the present application.

[0098] In an embodiment of the present application, labels can be divided into different types according to whether the label values ​​can be exhaustively enumerated and the dimension / quantity of the label values.

[0099] Exemplary, reference Figure 6 ,The label types in this application may include enumeration types, numeric types, date types, dimension table types, and nested types.

[0100] Among them, the enumeration type is also the dim type. The possible label values ​​of the enumeration type label can be expressed in an exhaustive form. For example, the possible label values ​​of the "gender" label shown in Table 1 are "male" or "female". In this way, the "gender" label can be an enumeration type label.

[0101] The numeric type is also called the number type. The possible tag values ​​of this numeric type tag can be exhaustively enumerated, but the number of optional tag values ​​is very large. For example, although the tag values ​​of the "Power-on Days" tag can be exhaustively enumerated, the exhaustive enumeration of possible tag values ​​will correspond to a large number of results.

[0102] The date type is also called the date type. The possible tag values ​​of the date type tag can also be exhaustively enumerated, but the amount of data in the set of possible tag values ​​is very large. That is, although it can be exhaustively enumerated, it is difficult to achieve. For example, the "activation date" tag can be a date type tag.

[0103] In this application, the enumeration type, the numerical type, and the date type may also be collectively referred to as other types. The tag values ​​of the other type tags can be exhaustively listed, and, in the behavior data, the number of tag values ​​of the other type tags is 1.

[0104] The dimension table type is also called array type. The number of label values ​​corresponding to the dimension table type label can be greater than 1. For example, the "installed APP" label in Table 1 can be a dimension table type label. The label value of the "installed APP" label can include at least 2 results such as "application 1" and "application 2".

[0105] In addition, the tag type may also include a nested type (mix type). The tag value of the nested type tag may include multiple values ​​of multiple dimensions.

[0106] Exemplarily, take the nested type tag including the "APP usage time" tag as an example. Correspondingly, the tag value of the "APP usage time" tag can include [{"app_name":"Application 1","app_use_duration":"3"}, {"app_name":"Application 2","app_user_duration":"7"}]. Thus, the behavior data including the mix type tag can be represented accordingly: under the "APP usage time" tag, "app_name" (i.e. application name) is "Application 1", "app_use_duration" (i.e. usage time) is "3"; "app_name" is "Application 2", "app_use_duration" is "7".

[0107] It can be seen that the tag value of the nested type tag can include the "app_name" dimension and the corresponding value, and the "app_use_duration" dimension and the corresponding value. That is, the tag value of the "APP usage duration" tag includes the tag values ​​of two dimensions.

[0108] The above examples illustrate the behavior data and label types in this application.

[0109] In this application, combined Figure 3 As described above, the configured condition group may include multiple tags. For example, each of the multiple tags may correspond to Figure 6 The array tag, mix tag, or other tags shown.

[0110] In different implementations, the different tags can be connected through the logic of and / or.

[0111] Exemplary, reference Figure 7 , taking the configuration of condition group CG71 in the cloud (such as the first processing module) as an example.

[0112] The condition group CG71 may include a label S71, a label S72, a label S73, a label S74, a label S75, a label S76, and a label S77.

[0113] The tag S71 is logically connected to the tag S72 through an "and" relationship, the tag S72 is logically connected to the tag S73 through an "and" relationship, and the tag S73 is logically connected to the tag S74 through an "or" relationship.

[0114] The tag S75 is logically connected to the tag S76 through an "or" relationship, and the tag S76 is logically connected to the tag S77 through an "and" relationship.

[0115] In this way, tag S71, tag S72, tag S73, and tag S74 can form a tag group. Tag S75, tag S76, and tag S77 can form another tag group. The judgment results of the two tag groups can be logically connected through an "and" relationship.

[0116] Therefore, through the various tags and the logical connections between the various tags, the condition group CG71 can be jointly formed.

[0117] It is understandable that in different implementations, the labels, the number of labels, and the logical connection relationship between the labels in the condition group CG71 can be configured. Figure 7 The composition of the condition group CG71 shown does not constitute a limitation to the present application. For example, the condition group can be configured with two or more tag groups, and each tag group can be configured with one or more tags.

[0118] The following combination Figures 5 to 7 The implementation of the solution provided in the embodiment of the present application is described below.

[0119] Exemplary, reference Figure 8 , is a schematic diagram of the module interaction flow of a data processing method provided in an embodiment of the present application. Figure 8The process shown can achieve efficient calculation of new groups.

[0120] like Figure 8 As shown, the program may include:

[0121] S801. The first processing module configures a new condition group CG81 and configures a population identifier C1 corresponding to the condition group CG81.

[0122] Exemplarily, the first processing module can display a UI configuration interface through a connected display device. Through the UI configuration interface, the first processing module can receive operations input by a developer. Correspondingly, the first processing module can obtain information about each label of the newly created condition group CG1 and the logical relationship between the labels based on the received operation information.

[0123] In this example, after receiving a complete newly created condition group CG81, the first processing module can configure a corresponding crowd identifier C1 for the newly created condition group. It can be understood that a condition group can correspond to a crowd information. In this way, the crowd identifier C1 can also be the crowd identifier corresponding to the crowd information of the condition group CG81.

[0124] In some embodiments of the present application, the first processing module may configure different crowd identifiers for different condition groups, so that a unique correspondence between a condition group / crowd information can be obtained through the crowd identifier.

[0125] As an example, the crowd identification can be a digital identification by adding "C" to 6. For example, the crowd identification of the newly created condition group CG81 can be configured as "C000001". For another example, the crowd identification of the condition group configured after the newly created condition group CG81 can be configured as "C000002", "C000003", etc.

[0126] S802: The storage module obtains the behavior data table BD1, and configures the state of the behavior data table BD1 to be a first value. In some embodiments, the behavior data table BD1 may also be referred to as a first behavior data table.

[0127] Exemplarily, the storage module may obtain user behavior data from the user's electronic device and / or an application server that is communicatively connected to the electronic device. For example, the user behavior data may include data of multiple tags and tag values ​​of each tag as shown in Table 1 or similar to Table 1.

[0128] As described in Table 1, an example of behavior data of a user is provided in Table 1. In this example, the behavior data table BD1 may include behavior data of multiple users. The behavior data of each user may include a user identifier, and at least one corresponding tag and tag value.

[0129] In the present application, each behavior data table can be configured with a status identifier. The status identifier can be configured as a first value or a second value. When the status identifier is configured as the first value, the data corresponding to the behavior data table has been configured / received, that is, it is in a ready state and can be called by other modules. Correspondingly, when the status identifier is configured as the second value, the data corresponding to the behavior data table has not been configured / received, that is, it is in the configuration and cannot be called by other modules.

[0130] In this example, it is taken that the behavior data table BD1 in the storage module has been configured, and the state flag is correspondingly configured as the first value as an example.

[0131] S803: The first processing module determines that the state identifier of the behavior data table BD1 is a first value.

[0132] Exemplarily, the first processing module can determine that the data in the behavior data table BD1 is available by determining that the state identifier of the behavior data table BD1 is the first value. In this way, the first processing module can control other modules to cooperate and configure the users in the behavior data table BD1 to the corresponding population information according to the condition group CG81.

[0133] Specifically, it can be achieved through the following steps.

[0134] S804: The first processing module sends a transmission instruction 81 to the second processing module.

[0135] Exemplarily, the first processing module may call the data transmission capability of the second processing module through the transmission instruction 81. The transmission instruction 81 is used to synchronize the data in the behavior data table BD1 to the computing module by calling the Spark engine.

[0136] In response to the transmission instruction 81 , the second processing module may sequentially read each behavior data in the behavior data table BD1 , and perform processing such as conversion and formatting on the read behavior data, and synchronize the data to the table item in the calculation module for storage.

[0137] It should be noted that, in the embodiment of the present application, when synchronizing the labels and label values ​​in each behavior data, the second processing module may perform different processing for different types of labels and label values.

[0138] The following steps S805-S811 are used to illustrate the different processing mechanisms. Take the behavior data table BD1 including label S81 information and label S82 information as an example. Label S81 information includes label S81 and the label value of label S81. The label S81 can be a mix type label. Label S82 information includes label S82 and the label value of label S82. The label S82 can be a non-mix type label. For example, the label S82 can be an array type label, or other types of labels.

[0139] S805. The second processing module obtains the label S81 information in the behavior data table BD1.

[0140] Exemplarily, the second processing module may obtain a piece of behavior information in the behavior data table BD1. The behavior information may include a user identifier and tag S81 information.

[0141] For example, the label of the tag S81 information may be "APP usage duration", and the tag value of tag S81 may include [{"app_name":"Application 1","app_use_duration":"3"}, {"app_name":"Application 2","app_user_duration":"7"}].

[0142] S806. The second processing module obtains sub-tag S81a information and sub-tag S81b information according to tag S81 information.

[0143] Exemplarily, the second processing module may split the label information of the mix type to obtain a sub-label information of each dimension. The length of each sub-label information is the same.

[0144] Take the label S81 information with the label "APP usage time" as an example.

[0145] The second processing module can split the "APP usage time" label information. The second processing module can obtain sub-label S81a information including: ["app_name":"Application 1","Application 2"}. The sub-label S81a information can be a single dimension sub-label corresponding to the "app_name" dimension.

[0146] In addition, the second processing module can also obtain sub-tag S81b information including: ["app_user_duration":"3","7"} through splitting processing. The sub-tag S81b information can be a single dimension sub-tag corresponding to the "app_user_duration" dimension.

[0147] In some embodiments, the sub-tag S81a information and the sub-tag S81b may also carry the user identification corresponding to the tag S81 respectively.

[0148] S807. The second processing module sends the sub-tag S81a information and the sub-tag S81b information to the calculation module.

[0149] S808. The calculation module stores the sub-tag S81a information and the sub-tag S81b information in the behavior data table BD2 respectively.

[0150] In the present application, a behavior data table BD2 may be configured in the computing module for storing behavior data synchronized from HIVE.

[0151] Exemplarily, the calculation module may store the label indicated by the sub-label S81a, the label value, and the corresponding relationship between the corresponding user identifier in a table entry of the behavior data table BD2.

[0152] The calculation module may also store the label indicated by the sub-label S81b, the label value, and the corresponding relationship between the corresponding user identifier in another table item of the behavior data table BD2.

[0153] That is to say, through the splitting and synchronization of the second processing module, a mix type of label data in the behavior data table BD1 can be stored in the behavior data table BD2 as two or more label data corresponding to different dimensions.

[0154] The above S805-S808 provide an example of a method for synchronizing mixed type tag data to a computing module. The following example of a synchronization process of non-mix type tag data is given in conjunction with S809-S811.

[0155] S809. The second processing module obtains the label S82 information in the behavior data table BD1.

[0156] Exemplarily, the tag S82 information may be tag information in a behavior data in the behavior data table BD1. In this way, the second processing module may obtain the corresponding user identification while obtaining the tag S82 information.

[0157] S810. The second processing module sends label S82 information to the calculation module.

[0158] S811. The calculation module stores the tag S82 information into the behavior data table BD2.

[0159] Exemplarily, when sending the tag S82 information, the second processing module may also send the user identifier corresponding to the tag S82 to the computing module. Thus, for non-mix type tag data, the second processing module may directly forward the tag data (such as tag S82 information) to the computing module for storage.

[0160] Correspondingly, the calculation module may store the label, label value, and corresponding user identification indicated by the label S82 information as a table entry in the behavior data table BD2.

[0161] In this way, the second computing module and the computing module can be configured with each other, and for the label information of each behavior data in the behavior data table BD1, S805-S808 or S809-S811 can be executed according to the label type to synchronize the label information of each behavior data in the behavior data table BD1 to the behavior data table BD2.

[0162] It is understandable that, based on the synchronization solution provided in the embodiment of the present application, the behavior data stored in the behavior data table BD2 in the calculation module can be single-dimensional label information. Compared with the calculation of a new population of mixed mix type label data and non-mix type label data, the calculation of label data of a single dimension can enable the calculation module to obtain higher calculation efficiency.

[0163] It is understandable that by executing the above steps, the synchronization process of the behavior data to the calculation module can be realized. The calculation module can also obtain the condition group CG81 through the following steps S812-S814.

[0164] S812. The first processing module rearranges the condition group CG81 to obtain the condition group CG811.

[0165] S813, the first processing module generates SQL11 according to the condition group CG811. In some embodiments, the SQL11 may also be referred to as the third SQL information.

[0166] In an embodiment of the present application, the first processing module can be used to generate corresponding control information according to the labels, label values ​​and logical relationships between the labels of the configured condition group CG81.

[0167] Exemplarily, the control information may include SQL11. That is, the SQL11 may correspond to the label, label value, and logical relationship between labels of CG81.

[0168] It should be noted that, in some embodiments, the first processing module can obtain corresponding control information by parsing according to the label, label value and logical relationship between labels of the condition group CG81 (ie, label information of the condition group CG81). The control information may include SQL11.

[0169] In other embodiments, Figure 8 As shown in S812-S813, the first processing module can also re-arrange the label information of the condition group CG81.

[0170] The rearrangement process may include: for each tag group, adjusting two or more tags of the mix type to the head of the tag group, adjusting the tags of the array type to the tag group, after the tags of the mix type, and adjusting other types of tags to after the tags of the array type.

[0171] It should be pointed out that, during the adjustment of the order of tags within a tag group, the logical relationship between tags remains unchanged.

[0172] As an example, refer to Fig. 9 , the composition of conditional group CG81 is similar to Figure 7 The condition group CG71 shown is the same as an example.

[0173] Among them, label S71, label S72, label S73, label S74, label S75, label S76, label S77.

[0174] Tag S71 is of array type, tag S72 is of mix type, tag S73 is of other types, and tag S74 is of mix type.

[0175] Tag S75 is of array type, tag S76 is of other types, and tag S77 is of mix type.

[0176] Tags S71, S72, S73, and S74 form a tag group. Tags S75, S76, and S77 form a tag group.

[0177] The first processing module can re-arrange the tags in the two tag groups respectively.

[0178] For example, take a tag group consisting of tag S71, tag S72, tag S73, and tag S74 as an example.

[0179] After rearrangement, the tag group can be adjusted to: tag S72, or, tag S74, or, tag S71, and, tag S73. In this way, in the tag group, the tag S72 and tag S74 of the mix type are adjusted to the head of the tag group. After the tag of the mix type, there can be a tag S71 of the array type. After the tag of the array type, there can be a tag S73 of another type.

[0180] For another example, take the tag group consisting of tag S75, tag S76, and tag S77 as an example.

[0181] After rearrangement, the tag group can be adjusted to: tag S77, and tag S75, or tag S76. In this way, in the tag group, the tag S77 of the mix type is adjusted to the head of the tag group. After the tag of the mix type, there can be a tag S75 of the array type. After the tag of the array type, there can be a tag S76 of other types.

[0182] In the present application, the rearrangement process can be performed within each tag group. Between tag groups, the original configuration logic can be maintained. Fig. 9 In the example, the two label groups obtained after the rearrangement process can be logically connected through the relationship of and, thereby obtaining the condition group CG811 after the rearrangement process.

[0183] It can be understood that, through the rearrangement processing in this example, the mix type tags in each tag group can be adjusted to the head of the tag group. As a result, when calculating according to this sequence, the mix type tags can be processed first, then the array type tags, and finally the other types of tags. Through actual verification, the calculation logic based on the rearrangement processing can effectively improve the calculation efficiency of crowd calculation (such as new crowd or existing crowd).

[0184] In this way, the first processing module can generate corresponding SQL11 according to the condition group CG811 after rearrangement of each tag group, so that the first processing module can control the calculation module to perform crowd calculation through the SQL11.

[0185] It is understandable that the SQL11 can be generated based on the rearranged condition group CG811. Therefore, when the calculation module performs calculations according to the SQL11, it can first perform calculations of the mix type in the tag group, then perform calculations of the array type in the tag group, and finally perform calculations of other types in the tag group.

[0186] S814. The first processing module sends SQL11 and the crowd identifier C1 to the calculation module.

[0187] The crowd identifier C1 may be the crowd identifier configured by the first processing module for the condition group CG81 after the first processing module completes configuring the condition group CG81 in S801.

[0188] S815. The first processing module obtains the meta information M1 of the condition group CG81, and stores a correspondence table BD3 between the meta information M1 and the crowd identifier C1.

[0189] In the embodiment of the present application, the meta information of the condition group may include at least one of the following:

[0190] The name of the population, the validity period of the condition group, the calculation period of the condition group, the tags in the condition group, and the tag values.

[0191] In other embodiments, the meta information of the condition group may further include the logical relationship between tags.

[0192] It is understandable that the meta information of the condition group may be input by the developer when configuring the condition group. That is, the meta information of the condition group may correspond to the characteristics of the current condition group (or the corresponding population).

[0193] In this example, the first processing module can instruct the calculation module to calculate the new crowd corresponding to the condition group of the crowd identifier C1 by executing S814. Therefore, after executing S814, the crowd information corresponding to the crowd identifier C1 can be obtained in the cloud.

[0194] Correspondingly, the first processing module can store the correspondence between the crowd identifier C1 and the meta information M1 in the correspondence table BD3, so that the subsequent calculation of the existing population of the crowd identifier C1 can be quickly implemented based on the meta information M1. The details will be described later.

[0195] Exemplarily, in addition to instructing the calculation module to calculate the new population corresponding to the condition group CG81 through SQL11, the first processing module can also store the metadata M1 corresponding to the condition group CG81 in the storage space configured in the first storage module (such as the cache of the first storage module).

[0196] As an implementation, the first processing module may be configured with a correspondence table BD3. The correspondence table BD3 may include a correspondence between at least one meta information and a crowd identifier. Through the correspondence, each module in the cloud can determine the correspondence between the meta information, the crowd information, and the condition group.

[0197] It is understandable that, based on the processing mechanism of S814 of the first processing module, the calculation module can obtain the judgment condition corresponding to the newly created condition group (such as condition group CG81). The judgment condition can be indicated by SQL11. The judgment condition can include multiple tags, tag values, and logical relationships between tags.

[0198] Thus, the calculation module can obtain SQL11 corresponding to S814 and the behavior data table BD2 including the behavior data corresponding to S805-S811. The calculation module can realize the calculation of the newly created group through the following steps.

[0199] S816. The calculation module calculates and obtains the crowd bitmap bitmap_1 corresponding to the condition group CG81 according to SQL11 and the behavior data table BD2.

[0200] It is understandable that since SQL11 is generated according to the rearranged condition group CG811, when the calculation module performs logical judgment according to SQL11, it can judge each label in each label group in the order of mix type, array type, and other types, thereby obtaining higher calculation efficiency.

[0201] In addition, since the behavior data stored in the behavior data table BD2 are all single-dimensional behavior data, the calculation efficiency can be further improved compared to directly calculating according to the behavior data table BD1 stored in HIVE.

[0202] In the embodiment of the present application, the calculation module can store the crowd information in the form of a bitmap.

[0203] In combination with the above description, the crowd information may include a collection of user identifiers that meet the current condition group. The user identifier may be the SN of the electronic device used by the user. Generally, the SN may include a combination of 13 digits and letters. Thus, when the amount of crowd information corresponding to the condition group CG81 is large (e.g., millions or more), if the crowd information is stored by storing the SN of each user, a very large storage space will be occupied.

[0204] In this example, each module in the cloud can share a bitmap correspondence table, which can be used to indicate the correspondence between different bits in the bitmap sequence and the user SN when different bits are configured as different numbers.

[0205] For example, when the first bit of the bitmap is configured as 1 (such as "0000000000001"), the corresponding user SN is SN1. When the second bit of the bitmap is configured as 1 (such as "0000000000010"), the corresponding user SN is SN2.

[0206] In this way, SN1 and SN2 can be identified correspondingly by the bitmap sequence "0000000000011".

[0207] Similarly, by configuring different bits in a bitmap sequence, it is possible to indicate whether the current crowd information includes the corresponding user, thereby enabling the calculation module to indicate the crowd information corresponding to the current condition group through a bitmap sequence.

[0208] Since the bitmap correspondence table can be shared by various modules in the cloud, each module can determine the user identifier specifically included in the crowd information based on the bitmap sequence and the bitmap correspondence table through the storage and transmission of the bitmap sequence.

[0209] As an example, the calculation module can calculate according to the label in each behavior data in the behavior data table BD2 and SQL11 to determine whether the label and label value of the behavior data meet the conditions corresponding to SQ11. If yes, in bitmap_1, the corresponding marking bit of the user identifier corresponding to the behavior data is configured as 1. Otherwise, in bitmap_1, the corresponding marking bit of the user identifier corresponding to the behavior data is configured as 0.

[0210] By analogy, the corresponding population bitmap (bitmap_1) sequence can be obtained according to SQL11 and behavior data table BD2.

[0211] S817. The calculation module stores a correspondence table BD4 between the crowd identifier C1 and the crowd bitmap bitmap_1.

[0212] In the embodiment of the present application, a corresponding table BD4 may be configured in the calculation module. The corresponding table BD4 may include one or more table entries. Each table entry stores a corresponding relationship between a crowd identifier and a crowd bitmap.

[0213] For example, after the calculation module obtains the crowd bitmap bitmap_1 corresponding to the crowd identifier C1 through calculation in S816, the corresponding relationship may be stored in the corresponding table BD4.

[0214] It is understandable that, in actual implementation, the calculation module can process multiple different condition groups simultaneously or in different time periods, so that the calculation module can obtain the corresponding relationship between multiple different crowd identifiers and crowd bitmaps.

[0215] Then, the calculation module can store the corresponding relationship between the different group identifiers and the corresponding bitmaps in different table entries of the corresponding table BD4.

[0216] In some embodiments, each entry in the corresponding table BD4 can be used to indicate population information of different condition groups.

[0217] In other embodiments, the cloud (such as the first processing module in the cloud) may also restore the bitmap in the corresponding table BD4 to a set of SNs for long-term storage and subsequent fast calling.

[0218] Exemplarily, the implementation of the solution may include S818-S821, wherein the restoration and storage of bitmap_1 of the crowd identifier C1 is taken as an example.

[0219] S818. The calculation module sends a correspondence table BD4 including the crowd identifier C1 and the crowd bitmap bitmap_1 to the second processing module.

[0220] Exemplarily, the process of S818 may be performed under the control of the first processing module.

[0221] As an example, the first processing module may call the second processing module to obtain the corresponding table BD4 from the calculation module.

[0222] Correspondingly, the calculation module can send the corresponding table BD4 to the second processing module through S818. In this example, the corresponding table BD4 may include the crowd bitmap bitmap_1 corresponding to the crowd identifier C1 determined by the calculation module, and the crowd identifier C1.

[0223] S819. The second processing module obtains corresponding crowd information PD1 according to the crowd bitmap bitmap_1.

[0224] Exemplarily, after the first processing module calls the second processing module to obtain bitmap_1, the first processing module may instruct the second processing module to perform restoration processing on the bitmap_1.

[0225] As a possible implementation, the second processing module can restore the bitmap_1 sequence to a corresponding set of one or more SNs according to the bitmap corresponding table in the above example. In this way, the set of one or more SNs can constitute the crowd information PD1.

[0226] Optionally, the second processing module can establish a correspondence between the crowd identifier C1 and the crowd information PD1. Thus, the newly created crowd corresponding to the newly created condition group CG81 can be obtained. The crowd identifier of the newly created crowd can be the crowd identifier C1, and the SN set indicated by the newly created crowd can include the set of SNs obtained by restoring the above bitmap_1 sequence.

[0227] S820. The second processing module sends the crowd identification C1 and the crowd information PD1 to the storage module.

[0228] S821. The storage module stores a correspondence table BD5 of the crowd identification C1 and the crowd information PD1.

[0229] In this example, a corresponding table BD5 may also be configured in the storage module. The corresponding table BD5 may be used to store the corresponding relationship between different group identifiers and specific SNs in the corresponding group information.

[0230] Exemplarily, the storage module may store the entry in the corresponding table BD5 according to the received crowd identifier C1 and crowd information PD1, thereby realizing the calculation and persistent storage of the newly created crowd corresponding to the condition group CG81.

[0231] It is understandable that in this Figure 8 In the implementation of the scheme shown, the first processing module can control the second processing module to split the mix tag in the behavior data into single-dimensional behavior data and synchronize it to the calculation module, thereby improving the calculation efficiency of the calculation module. In addition, the first processing module can re-arrange the configured condition groups, so as to obtain the corresponding SQL according to the re-arrangement results and send it to the calculation module, so that the calculation module can prioritize the judgment of the mix type based on the re-arranged SQL when executing the judgment in each condition group, thereby further improving the calculation efficiency. In addition, the calculation module can also store crowd information through bitmap, thereby saving storage space in the calculation module.

[0232] Thus, through Figure 8 The solution shown can achieve efficient calculation of new crowds.

[0233] After the calculation of the newly created group is completed, a corresponding table BD3 may be stored in the first processing module, including the corresponding relationship between the condition group CG81 and the meta information M1.

[0234] In the embodiment of the present application, when calculating the existing population of the population information corresponding to the condition group CG1, the population information can be updated quickly and efficiently based on the metadata in the corresponding table BD3.

[0235] It is understandable that since the crowd information is determined based on the user's behavior data, the user's behavior data will change over time. For example, the number of user samples changes (such as increase, replacement, etc.); or the change of the same user's behavior data. These changes in user behavior data will affect the crowd information of the same condition group.

[0236] Combined with the above description, in this application, each module in the cloud can calculate the stock population of the same condition group according to the relevant information of the condition group configuration (such as validity period, calculation cycle, etc.). The calculation of the stock population can also be understood as the process of determining the corresponding population information of the condition group based on the existing condition group and the updated behavior data.

[0237] It is understandable that before calculating the existing population, the calculation process of the newly created population must be carried out. Exemplarily, the calculation process of the newly created population may correspond to the following: Figure 8 The scheme shown is implemented. Thus, the corresponding table BD3 of the first processing module can store the table items corresponding to the stock population. For example, the calculated stock population corresponding condition group is condition group CG81. Figure 8 The solution shown is implemented so that the correspondence between the conditional group CG81 and the meta-information M1 can be stored in the correspondence table BD3 in the first processing module.

[0238] Based on this, the embodiment of the present application also provides a solution that can achieve efficient calculation of the existing population based on the corresponding table BD3.

[0239] Combined with the above description of the corresponding table BD3, in the embodiment of the present application, when the cloud can be configured with multiple condition groups, the corresponding table BD3 can include table items corresponding to multiple different condition groups, wherein each table item can include a corresponding relationship between a population identifier and meta information.

[0240] For example, the BD3 may include a correspondence between a crowd identifier C1 and meta information M1.

[0241] In this application, each module in the cloud can cooperate with each other through Fig.10 The solution shown in the figure is implemented by generating a population bitmap corresponding to each used tag in the calculation module. The population bitmap corresponding to each tag can be used to calculate the stock population of other condition groups. In this example, the usage behavior data tables of each condition group are as follows Figure 8 The behavior data table BD1 shown is taken as an example.

[0242] like Fig.10 As shown, the program may include:

[0243] S1001. The storage module updates the behavior data table BD1, and the state identifier of the behavior data table BD1 is a second value.

[0244] S1002: The storage module completes updating the behavior data table BD1, and the status identifier of the behavior data table BD1 is the first value.

[0245] Exemplarily, the storage module can directly or indirectly obtain user behavior data, thereby updating the behavior data table BD1. While the behavior data table BD1 in the storage module is being updated, the status identifier of the behavior data table BD1 can be configured to a second value to indicate that the behavior data table BD1 is unavailable. Correspondingly, after the behavior data table BD1 is updated, the status identifier of the behavior data table BD1 in the storage module can be changed to a first value, indicating that the behavior data table BD1 is available, or that the behavior data table BD1 is in a ready state.

[0246] S1003: The first processing module determines that the state identifier of the behavior data table BD1 is a first value.

[0247] Exemplarily, the execution of this step can refer to Figure 8 S802 is shown. In this way, the first processing module can determine to execute subsequent steps to achieve update synchronization of the information in the behavior data table BD1 to the calculation module.

[0248] S1004: The first processing module sends a transmission instruction 82 to the second processing module. In some embodiments, the transmission instruction 82 may also be referred to as a first transmission instruction.

[0249] For example, the execution of S1004 can refer to Figure 8 The specific implementation of S803 is not described in detail.

[0250] S1005. The second processing module obtains label information in the behavior data table BD1 from the storage module.

[0251] S1006: The second processing module synchronizes the tag information to the behavior data table BD6 configured in the calculation module. In some embodiments, the behavior data table BD6 may also be referred to as a first behavior data table.

[0252] For example, the execution of S1005-S1006 can refer to Figure 8 Description of S805-S811 shown.

[0253] As a possible implementation, the second processing module may synchronize each behavior data in the updated behavior data table BD1 to the calculation module by executing S1005 - S1006 .

[0254] In this example, a behavior data table BD6 may be configured in the calculation module to store the acquired behavior data.

[0255] Combination Figure 8As described in, the execution process of S1005-S1006 may include the splitting and synchronization of the mix type tags, and the synchronization of the non-mix type tags. Thus, the updated behavior data may be stored in the behavior data table BD6. Each behavior data in the behavior data table BD6 may include a single-dimensional tag and a tag value.

[0256] S1007: The first processing module generates SQL12 according to the corresponding table BD3. In some embodiments, SQL12 may also be referred to as second SQL information. The corresponding table BD3 may also be referred to as a second comparison table.

[0257] Exemplarily, the first processing module may generate SQL12 according to one or more table entries in the corresponding table BD3.

[0258] In some embodiments, the first processing module may generate SQL12 according to the label information in the corresponding table BD3.

[0259] For example, the first processing module can generate SQL12 independently for the labels and label values ​​indicated by each label information used in the corresponding table BD3 (i.e., all label information included in the corresponding table BD3), so that other modules (such as the calculation module) can calculate each label and label value according to SQL12 to obtain the population information corresponding to each label information.

[0260] In some implementations, the first processing module may sort the respective tag information by popularity, and configure the tag information with higher popularity at the head of SQL12, and correspondingly, configure the tag information with lower popularity at the back of SQL12.

[0261] As an example, it is taken that the label information corresponding labels used in the correspondence table BD3 include: label S71, label S72, label S73, label S74, label S75, label S76, and label S77.

[0262] The first processing module may perform a popularity ranking on each tag according to the number of occurrences of the tag information.

[0263] For example, the number of occurrences of each tag from high to low is: tag S72, tag S73, tag S71, tag S75, tag S77, tag S74, tag S76.

[0264] Correspondingly, the popularity ranking of each tag from high to low is: tag S72, tag S73, tag S71, tag S75, tag S77, tag S74, tag S76.

[0265] Therefore, the first processing module can generate SQL12 according to the heat sorting.

[0266] Exemplarily, in SQL12, the order of label information indicating the judgment is: label information of label S72, label information of label S73, label information of label S71, label information of label S75, label information of label S77, label information of label S74, and label information of label S76.

[0267] S1008. The first processing module sends SQL12 to the calculation module.

[0268] S1009. The calculation module calculates and obtains the crowd bitmap corresponding to each tag according to SQL12 and the behavior data table BD6.

[0269] S1010. The calculation module stores a corresponding table BD7, including a plurality of labels, label values, and corresponding relationships between corresponding population bitmaps.

[0270] In combination with the description in S1007, the calculation module may perform calculations corresponding to each tag according to the sequence of each tag information indicated in SQL12.

[0271] Since each label is independent, the result calculated according to SQL12 may include a population bitmap corresponding to each label.

[0272] For example, the calculation module can obtain the bitmaps corresponding to the tags S72, S73, S71, S75, S77, S74, and S76 respectively according to the behavior data table BD6.

[0273] As an example, a corresponding table BD7 may be configured in the calculation module. Each entry in the corresponding table BD7 is used to store a corresponding relationship between a label, a label value and a crowd bitmap.

[0274] For example, the following Table 2 provides an example of a corresponding table BD7.

[0275] Table 2

[0276] Label Encoding Tag value Bitmap Tag table Tag S72 Value_1 Crowd Position Figure 1 Tag Table 1 Tag S73 Value_2 Crowd Position Figure 2 Tag Table 2 …… …… …… ……

[0277] As shown in Table 2, the corresponding table BD7 may include a population bitmap corresponding to each tag information.

[0278] Each of the tag information may include a tag and a tag value. For example, the tag information of tag S72 includes a tag code of tag S72 and a tag value of Value_1. For another example, the tag information of tag S73 includes a tag code of tag S73 and a tag value of Value_2.

[0279] The corresponding table BD4 may also include a population bitmap corresponding to each tag information. The SN corresponding to the population bitmap may be obtained based on the updated and synchronized behavior data in the behavior data table BD6.

[0280] In some embodiments, as shown in Table 2, the corresponding table BD7 may include label tables described in various label information. Different label tables correspond to different condition groups.

[0281] In this way, a plurality of bitmaps with labels as granularity can be obtained in the calculation module based on the updated behavior data, and each of the plurality of bitmaps can correspond to a label in the existing condition group.

[0282] Based on the corresponding table BD7, the stock population of each condition group can be quickly calculated.

[0283] Exemplary, reference Fig.11 , is a flow chart of a data processing method provided in an embodiment of the present application. Fig.11 The solution shown in the figure can be used to calculate the population of a conditional group. Fig.11 The scheme shown takes the calculation of the existing population for condition group CG81 as an example.

[0284] S1101, the first processing module generates SQL13 according to the configured condition group CG81. In some embodiments, SQL13 may also be referred to as first SQL information.

[0285] In this example, the first processing module can generate SQL13 according to the label information in the condition group CG81 and the connection relationship between each label.

[0286] It should be noted that, in the step shown in S1101, the first processing module does not need to rearrange the condition group CG81. The first processing module can directly generate SQL13 according to the configured CG81.

[0287] S1102. The first processing module sends SQL13 and the crowd identifier C1 to the calculation module.

[0288] S1103, the calculation module calculates and obtains the crowd bitmap bitmap_2 corresponding to the condition group CG81 according to SQL13 and the corresponding table BD7. In some embodiments, the corresponding table BD7 may also be referred to as a first comparison table.

[0289] In this example, SQL13 may instruct the calculation module to calculate and obtain the corresponding population bitmap according to a plurality of different tag information and the connection relationship between the tags.

[0290] Combination Fig.10 According to the description in, the corresponding table BD7 may include the population bitmaps corresponding to all the currently used tag information. In this way, the calculation module can obtain the population bitmaps of each tag from the corresponding table BD7 according to the tag information indicated by SQL13. The calculation module can also perform logical operations on the obtained population bitmaps according to the logical connection relationship of each tag information indicated by SQL13, and obtain the population bitmap bitmap_2 corresponding to the condition group CG81.

[0291] As an example, combining Fig. 9 The description of condition group CG81 is as follows Fig. 9 The 7 tags shown are taken as an example.

[0292] In combination with the description of the acquisition mechanism of the correspondence table BD7, the correspondence table BD7 may include the corresponding population bitmaps of the labels S71 to S77.

[0293] For example, tag S71 corresponds to bitmap_71, tag S72 corresponds to bitmap_72, tag S73 corresponds to bitmap_73, tag S71 corresponds to bitmap_74, tag S75 corresponds to bitmap_75, tag S76 corresponds to bitmap_76, and tag S77 corresponds to bitmap_77.

[0294] Therefore, the calculation module can obtain bitmap_71 to bitmap_77 respectively from the corresponding table BD7 according to SQL13.

[0295] The calculation module may also obtain the logical connection relationship of each bitmap according to SQL 13. The logical connection relationship of each bitmap may be the same as the logical connection relationship of the corresponding tag information.

[0296] The calculation module can perform logical operations on each bitmap according to the determined connection relationship, and the result can be the crowd bitmap bitmap_2 corresponding to the condition group CG81.

[0297] Exemplarily, the calculation module may perform an “AND” operation on bitmap_71 and bitmap_72, and perform an “AND” operation on bitmap_73, and perform an “OR” operation on bitmap_74, thereby obtaining the crowd bitmap of the first tag group.

[0298] The calculation module may perform an “OR” operation on bitmap_75 and bitmap_76, and perform an “AND” operation on bitmap_75 and bitmap_77, thereby obtaining the crowd bitmap of the second tag group.

[0299] The calculation module may also perform an “AND” operation on the crowd bitmap of the first tag group and the crowd bitmap of the second tag group, thereby obtaining the crowd bitmap bitmap_2.

[0300] S1104. The calculation module stores a correspondence table BD8 between the crowd identifier C1 and the crowd bitmap bitmap_2.

[0301] In this example, the calculation module may store the calculation results of the existing population in the corresponding table BD8. In some embodiments, the corresponding table BD8 and the corresponding table BD4 of the newly created population may be different partitions of the same table.

[0302] Similar to Figure 8 During the execution process, the corresponding relationship between the crowd bitmap and the crowd identifier can be stored in the calculation module.

[0303] In some of the above, each module in the cloud can also restore the crowd bitmap into a set of SNs, and store them together with the crowd identifiers in the storage module.

[0304] S1105. The calculation module sends a correspondence table BD8 of the crowd identifier C1 and the crowd bitmap bitmap_2 to the second processing module.

[0305] S1106. The second processing module obtains corresponding crowd information PD2 according to the crowd bitmap bitmap_2.

[0306] S1107. The second processing module sends the crowd identification C1 and the crowd information PD2 to the storage module.

[0307] S1108. The storage module stores a correspondence table BD9 of the crowd identification C1 and the crowd information PD2.

[0308] The exemplary execution process of S1105-S1108 can refer to Figure 8 S818-S821 in the figure will not be described in detail.

[0309] Thus, through Fig.11 The solution shown is implemented based on Fig.10 The population bitmap of each label obtained can quickly calculate the stock population corresponding to each condition group without having to re-judge the information of each label in each condition group based on the updated behavior data.

[0310] It is understandable that for other configured condition groups, each module in the cloud can also be based on the following Fig.11 A similar solution is implemented to quickly calculate the corresponding existing population.

[0311] In this implementation, due to Fig.10When obtaining the population bitmap corresponding to each tag information, only one calculation is required for each tag information to obtain the corresponding population bitmap. Compared with the general stock population calculation process, the repeated calculation of each tag in each stock population calculation can significantly save the corresponding overhead and improve efficiency. Furthermore, due to Fig.10 The strategy of calculating the population bitmap of the label information according to the popularity sorting makes the population bitmap corresponding to the label information with higher popularity configured into the corresponding table BD7 earlier. As a result, the population bitmap of the label information with higher popularity can be reused by more stock population calculation processes, further improving the calculation efficiency.

[0312] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of each functional module. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0313] The above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0314] For example, Fig.12 FIG. 1 is a schematic diagram showing the composition of a server device 1200. The server device 1200 may correspond to Figure 5 The first processing module is shown.

[0315] like Fig.12 As shown, the server device 1200 may include: a processor 1201 and a memory 1202. The memory 1202 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1201 executes the instructions stored in the memory 1202, the server device 1200 may execute the method corresponding to the first processing module in the above embodiment.

[0316] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0317] Fig.13A schematic diagram of the composition of a chip system 1300 is shown. The chip system 1300 may include: a processor 1301 and a communication interface 1302, which are used to support related devices to implement the functions involved in the above embodiments. In a possible design, the chip system also includes a memory for storing the necessary program instructions and data for the first processing module. The chip system can be composed of chips, and can also include chips and other discrete devices. It should be noted that in some implementations of the present application, the communication interface 1302 may also be referred to as an interface circuit. In some implementations, the chip system 1300 can be configured in the first processing module to support the related functions provided by the first processing module in any of the above embodiments.

[0318] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0319] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0320] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: Applied to a data processing device, the data processing device is configured with a first processing module and a computing module; Wherein, at least one condition group is configured in the first processing module, each of the condition groups includes at least one tag information, the tag information includes a tag, and the tag is configured with a tag value; The method comprises: The first processing module sends a first SQL message to the calculation module, where the first SQL message is used to indicate all label information in a first condition group and a logical connection relationship between each label information configured in the first condition group; the first condition group is included in the at least one condition group; The calculation module determines a first population bitmap according to the first comparison table and the first SQL information; the first population bitmap corresponds to the first condition group; The first comparison table includes a population bitmap corresponding to each tag information in the first condition group; The method further comprises: The first processing module sends a third SQL message to the calculation module, where the third SQL message is used to indicate all the label information in the second condition group and the logical connection relationship between each label information configured in the second condition group; The second condition group is obtained after the first processing module rearranges each label in the first condition group; The rearrangement process includes: for each label group in the first condition group, adjusting one or more labels of the nested mix type to the head of the label group, and adjusting one or more labels of the dimension group array type or other types to the tail of the label group; the label information of the mix type includes label information of at least two dimensions; The calculation module determines the second population bitmap corresponding to the first condition group according to the third SQL information and the second behavior data table before updating stored in the storage module.

2. The method according to claim 1, characterized in that After determining the first group of people information, the method further includes: The calculation module stores a correspondence between a first group identifier and the first group bitmap; the first group identifier corresponds to the first condition group.

3. The method according to claim 2, characterized in that The data processing device is further configured with a second processing module and a storage module, and the method further comprises: The first processing module controls the second processing module to obtain a correspondence between the first group identifier and the first group bitmap; The first processing module controls the second processing module to obtain a first user identification set according to the first population bitmap; The first processing module controls the second processing module to send the first user identification set and the first group identification to the storage module.

4. The method according to claim 3, characterized in that The method further comprises: The first processing module controls the storage module to store the correspondence between the first group identifier and the first user identifier set.

5. The method according to claim 3 or 4, characterized in that: Before the first processing module sends the first SQL information to the calculation module, the method further includes: The calculation module obtains the first comparison table under the control of the first processing module.

6. The method according to claim 5, characterized in that The calculation module, under the control of the first processing module, obtains the first comparison table, including: The first processing module controls the calculation module to obtain a first behavior data table; The first behavior data table includes at least one behavior data, and the behavior data in the first behavior data table corresponds to the behavior data in the second behavior data table; the second behavior data table is a behavior data table configured in the storage module; the second behavior data table includes at least one behavior data, and the behavior data includes a user identifier and at least one tag information corresponding to the user identifier; The first processing module controls the calculation module to obtain the first comparison table according to the first behavior data table.

7. The method according to claim 6, characterized in that The first processing module controls the calculation module to obtain a first behavior data table, including: After the first processing module completes updating the second behavior data table, the first processing module sends a first transmission instruction to the second processing module; In response to the first transmission instruction, the second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the calculation module.

8. The method according to claim 7, characterized in that The second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the calculation module, including: The second processing module obtains second behavior data from the second behavior data table, where the second behavior data includes first label information, and a type of the first label information is a nested mix type; The second processing module splits the first label information to obtain first sub-data and second sub-data, where the first sub-data and the second sub-data respectively represent label information of different dimensions in the first label information, and both the first sub-data and the second sub-data are label information of a single dimension; The second processing module stores the first sub-data and the second user identifier as a first table item of the first behavior data table; The second processing module stores the second sub-data and the second user identifier as a second table item of the first behavior data table; The second user identifier is a user identifier corresponding to the second behavior data.

9. The method according to claim 7 or 8, characterized in that: The second processing module synchronizes each behavior data in the second behavior data table to the first behavior data table of the calculation module, including: The second processing module obtains third behavior data from the second behavior data table, where the third behavior data includes second label information, and the type of the second label information is non-mix type; The second processing module stores the correspondence between the second tag information and the third user identifier in a third table item of the first behavior data; The third user identifier is a user identifier corresponding to the third behavior data.

10. The method according to claim 9, characterized in that The non-mix type tag information includes: Dimension group array type label information, and / or other types of label information; Each non-mix type of label information includes label information of one dimension.

11. The method according to any one of claims 6 to 8 or 10, characterized in that: The first processing module controls the calculation module to obtain the first comparison table according to the first behavior data table, including: The first processing module generates second SQL information, wherein the second SQL information includes all configured tag information; The first processing module sends the second SQL information to the calculation module; The calculation module determines, according to the second SQL information and the first behavior data table, a population bitmap corresponding to each of the label information indicated by the second SQL information; The calculation module stores the correspondence between each tag information indicated by the second SQL information and the corresponding population bitmap in the first comparison table.

12. The method according to claim 11, characterized in that The first processing module stores a second comparison table, wherein the second comparison table stores a correspondence between at least one group identifier and meta information; The correspondence between at least one group identifier and meta information includes: the correspondence between the first group identifier and the first meta information; the first meta information includes the label information indicated by the first condition group; The first processing module generates second SQL information, including: The first processing module generates the second SQL information according to the second comparison table.

13. A data processing device, characterized in that: The data processing device is configured with a first processing module, a second processing module, a storage module, and a computing module; the data processing device is used to determine the first population information corresponding to the first condition group according to the method described in any one of claims 1-12; the first population information includes a first population bitmap, and / or the first population information includes a first user identification set.

14. A server device, characterized in that: The server device is configured with a first processing module, and when the server device is working, it is used to implement the function of the first processing module in any one of claims 1-12.

15. A chip system, characterized in that: The chip system is configured in a server device as described in claim 14; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected via lines; the interface circuit is used to receive a signal from a memory and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the server device is used to implement the function of the first processing module in the method as described in any one of claims 1-12.

Images