Data processing method, device, storage medium and electronic device

Abstract

The present invention discloses a data processing method, device, storage medium and electronic device. The method includes: determining a target tag and a target trigger condition corresponding to the target tag; determining target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to at least one target object. Through the present invention, the technical effect of reducing the difficulty of formulating a real-time reach strategy is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data processing method, device, storage medium and electronic device. Background Art

[0002] Currently, in business operations, reaching out to users can maintain long-term contact with them. As the business continues to develop, there are more and more business scenarios that require real-time reaching of users, and the requirements for timeliness of reaching out are becoming higher and higher.

[0003] In related technologies, when achieving real-time reach, it is necessary to involve big data components such as open source processing platforms (Kafka), real-time stream processing frameworks (Spark-streaming) or open source processing frameworks (Flink). The operation threshold is high, and for most businesses and operations, there is no coding ability, so it is impossible to independently formulate some real-time reach strategies.

[0004] With regard to the technical problem of the difficulty in formulating real-time reach strategies in existing technologies, no effective solution has been proposed so far. Summary of the invention

[0005] The main purpose of the present invention is to provide a data processing method, device, storage medium and electronic device to at least solve the technical problem of the difficulty in formulating a real-time reach strategy.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present invention, a data processing method is provided. The method may include: determining a target tag and a target trigger condition corresponding to the target tag; determining target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object.

[0007] Optionally, determining the target tag based on the first log includes: acquiring a target field in the first log; and determining a value of the target field as the target tag.

[0008] Optionally, the method also includes: storing the target tag to the first target database through the server; sending a verification request to the server, wherein the verification request is used to request the server to verify the target tag in the first target database, and if the number of second logs including the target tag is greater than or equal to the target threshold, the target tag is successfully verified by the server.

[0009] Optionally, determining the target trigger condition corresponding to the target tag includes: determining the target trigger condition based on the successfully verified target tag.

[0010] Optionally, the method further includes: displaying configuration information of at least one target tag and / or a verification result of the target tag.

[0011] Optionally, when there are multiple target tags, determining the target trigger condition corresponding to the target tag includes: performing a logical combination on the multiple target tags to obtain the target trigger condition.

[0012] Optionally, the method further includes: determining that when the behavior data of at least one target object in the first target object set meets the target trigger condition, pushing target push data to the at least one target object.

[0013] Optionally, the method further includes: determining that when the behavior data of at least one target object in the second target object set meets the target trigger condition, pushing the target push data to the at least one target object is prohibited.

[0014] Optionally, the method also includes: storing the target policy data in a second target database through a server; sending a detection request to the server, wherein the detection request is used to request the server to detect whether the behavior data meets the target trigger conditions stored in the second target database, wherein the target policy data includes the target trigger conditions.

[0015] Optionally, the method further includes: displaying target policy data and / or target push data.

[0016] In order to achieve the above-mentioned object, according to another aspect of the present invention, a data processing device is provided. The device may include: a first determination unit, configured to determine a target tag and a target trigger condition corresponding to the target tag, wherein each target tag is used to represent information of a target field in a first log; and a second determination unit, configured to determine target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object.

[0017] To achieve the above object, according to another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, wherein the computer program is configured to execute the data processing method of the embodiment of the present invention when it is run.

[0018] To achieve the above object, according to another aspect of the present invention, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the data processing method of the embodiment of the present invention.

[0019] In an embodiment of the present invention, by determining a target tag and a target trigger condition corresponding to the target tag, wherein each target tag is used to represent the information of the target field in the first log; determining target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to at least one target object. In other words, this embodiment determines the target trigger condition based on the configured target tag, and then determines the target policy data based on the target trigger condition, thereby achieving the purpose of giving the business autonomous custom configuration of real-time trigger conditions, and real-time detection and reach of user behavior, solving the technical problem of the difficulty of formulating real-time reach strategies in the prior art, and achieving the technical effect of reducing the difficulty of formulating real-time reach strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 It is a hardware structure block diagram of a mobile terminal of a data processing method according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a target tag according to an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of another target tag according to an embodiment of the present invention;

[0025] Figure 5 is a flow chart of verifying a target tag according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of target strategy data according to an embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of a target tag configuration information interface display according to an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of another target strategy data according to an embodiment of the present invention;

[0029] Fig. 9 is a schematic diagram of a Flink real-time detection program sample according to an embodiment of the present invention;

[0030] Fig.10 is a schematic diagram of detecting real-time behavior of massive users based on Flink according to an embodiment of the present invention;

[0031] Fig.11 is a schematic diagram of a data processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] With the continuous development of operational business, in order to adapt to more and more business scenarios that require real-time user contact, the existing solutions mainly adopt pre-development solutions and data warehouse structured query language (Hive SQL) solutions for real-time contact.

[0036] The implementation process of the pre-developed solution is mainly as follows: business personnel summarize business requirements according to the business scenarios that need to be pushed, planners and technical personnel discuss specific solutions based on the summarized business requirements, technical personnel implement the solution technically, and after the quality assurance (QA) test of the technically implemented solution is completed, the solution can be launched. This pre-developed solution has good timeliness, but the implementation link of the entire solution is long, requires pre-development, has high R&D costs, and can only be pushed for specific business scenarios, which cannot meet the requirements of the business's autonomous customization of real-time reach strategies.

[0037] The implementation process of the Hive SQL solution is as follows: the technical staff stores the user behavior data in Hive in near real time, and the business staff selects the user list that meets the push requirements through specific SQL, and uploads the user list to the push system for push. In this Hive SQL solution, business staff can customize SQL to select the user list that needs to be pushed, but the implementation data link of the entire solution is long, and the data needs to be transmitted on different storage media, which has poor timeliness. Among them, timeliness refers to the delay between the user's real behavior and the actual implementation of the policy.

[0038] Existing solutions have the technical problem of making it difficult to formulate real-time reach strategies, which makes the operational threshold for real-time reach high.

[0039] In view of the above problems, the method embodiments provided in the application embodiments can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0040] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for data processing in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] In this embodiment, a data processing method running on the above mobile terminal is provided. Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0043] Step S22: determine the target tag and the target trigger condition corresponding to the target tag.

[0044] In the technical solution provided in the above step S22 of the present invention, the target tag is used to represent the information of the target field in the first log.

[0045] In this embodiment, the first log may be a log related to user operation behavior, for example, a user login log, which is not specifically limited here. In this embodiment, the target field information in the first log may be represented by a target tag, for example, the first log is a user login log LoginRole, which may include a JSON field, which contains many key-values. Figure 3 is a schematic diagram of a target tag according to an embodiment of the present invention. Figure 3As shown, the target tag may include the role level logged in, the application channel logged in, the application version logged in, the operating system logged in, etc., which are not specifically limited here. Among them, the target tag can also be called a real-time tag.

[0046] When the business formulates a strategy, for example, it can formulate what conditions some real-time tags of the login should meet. In essence, it is equivalent to that the fields corresponding to the log must meet the corresponding conditions.

[0047] As an optional implementation, in step S22, determining the target tag includes: determining the target tag based on the first log.

[0048] In this embodiment, the business can customize the target label on the graphical user interface according to actual needs. The first log can be obtained first. For the first log, the information of the target field corresponding to the business needs can be determined in the first log, and then the information of the target field can be determined as the target label, thereby realizing the configuration of the customized target label on the front end.

[0049] As an optional implementation, determining the target tag based on the first log includes: acquiring a target field in the first log; and determining a value of the target field as the target tag.

[0050] In this embodiment, the target field can have different values. After determining the target field corresponding to the business requirement in the first log, the value of the target field can be obtained. For example, if the target tag is the logged-in role level, the value 89 of the role level can be determined as the above-mentioned target tag.

[0051] As an optional implementation, the data processing method further includes the following steps:

[0052] Step 1: Store the target tag in the first target database through the server.

[0053] In this embodiment, the front end can write the target tag of the service configuration into the target tag table of the first target database for storage via the interface provided by the server, wherein the server can be a back-end server. Figure 4 FIG. 1 is a schematic diagram of another target tag according to an embodiment of the present invention. Figure 4 As shown, the first target database can be a target tag table, which can record multiple different target tags, such as the name, creator, call deadline, verification status, and operation options for the target tags of target tag 1, target tag 2, target tag, and target tag 4.

[0054] Step 2: Send a verification request to the server.

[0055] In this embodiment, the verification request is used to request the server to verify the target tag in the first target database. Optionally, if the number of second logs including the target tag is greater than or equal to the target threshold, the target tag is successfully verified by the server.

[0056] This embodiment can detect in real time in the server whether there is a second log including a target tag in the real log stream, that is, detect whether there is a log with data that meets the configuration corresponding to the target tag. If there is the above second log, it is determined that there is a target field corresponding to the target tag, and it is determined that one verification is successful. If the number of second logs including the target tag is greater than or equal to the target threshold, the target tag is successfully verified by the server, and the verification can be exited.

[0057] In this embodiment, the second log is a real user behavior log, which may be a log stream read in real time, for example, a log stream of a game read in real time based on Flink.

[0058] Optionally, if the number of second logs including the target tag is less than the target threshold, verification can be continued, and the verification time of the target tag can be further compared with the preset time. If the verification time of the target tag is greater than or equal to the preset time, it is determined that each target tag has timed out from verification by the server, and the target tag exits verification; if the verification time of the target tag is less than the preset time, the number of second logs including the target tag can be continued to be compared with the target threshold, and the above steps are repeated.

[0059] Optionally, this embodiment can deploy a Flink real-time label verification program (running in real time in the background) on the data side, which will consume Kafka data in real time and verify in real time whether the target label configured by the business independently meets the actual requirements of the data. For example, it mainly verifies whether the target label defined by the business (corresponding to a target field of a first log) actually exists in the real user behavior log.

[0060] For example, in the user login log, for the target tag, the business defines a login role level, but when configuring the page, it should be "Table: loginrole, Field: role_level". The business may have mistakenly configured it as "Table: login, Field: rolelevel". According to this configuration of the business, the specific value of the login role level cannot be found. Therefore, this embodiment needs to verify the target tag configured by the business, and the program can get the corresponding real value based on its configuration. Only target tags that have been successfully verified can be used in combination when formulating policies to form target trigger conditions.

[0061] Figure 5 FIG. 1 is a flow chart of verifying a target tag according to an embodiment of the present invention. Figure 5 As shown, the process is to establish a real-time target tag, put the target tag at the end of the verification queue, and obtain the real-time game log stream from Kafka, which includes the second log of the target tag. Determine whether the number of second logs is greater than or equal to the target threshold. When the result is yes, it is determined that the target tag verification is successful; when the result is no, it can be further determined whether the verification time is greater than or equal to the preset time. When the verification time is greater than or equal to the preset time, it is determined that the target tag verification has timed out. Otherwise, the number of second logs can be compared with the target threshold and the above steps can be repeated.

[0062] As an optional implementation manner, determining the target trigger condition corresponding to the target tag includes: determining the target trigger condition based on the successfully verified target tag.

[0063] In this embodiment, only successfully verified target tags can be used by the real-time strategy to determine the target triggering condition.

[0064] As an optional implementation, when there are multiple target tags, step S22, determining the target trigger condition corresponding to the target tag, includes: logically combining the multiple target tags to obtain the target trigger condition.

[0065] In this embodiment, the business can define multiple target tags on the front-end graphical user interface, and multiple target tags can be logically combined according to the target logical relationship. For example, the logical relationship can be "and", "or", or "=", "!=", ">", ">=", "<", "<=", including, not including, in, not in, etc., and no specific restrictions are made here. Optionally, this embodiment logically combines multiple target tags that have been successfully verified to obtain the above-mentioned target trigger condition.

[0066] Step S24, determining target policy data based on the target trigger condition.

[0067] In the technical solution provided in the above step S24 of the present invention, the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object.

[0068] In this embodiment, the target policy data may be all data related to the policy, such as the selected group, blacklist, trigger condition, maximum release volume, push content, etc. configured for the policy, and may also include policy identification, policy name, creator, policy classification, life cycle, creation date, release status, grayscale mechanism and progress, release volume and success volume, etc. The target policy data includes the target trigger condition and the corresponding target push data. When the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition can be pushed to the at least one target object.

[0069] For example, the above-mentioned target strategy data of this embodiment can be some business-developed game applications. When the user plays certain types of gameplay in real time and meets the target trigger conditions, the target push data in the target strategy data will be pushed. For example, if the user fails in a certain gameplay and meets the target trigger conditions, strategy data will be pushed to him, or if the user plays a certain gameplay and meets the target trigger conditions, some magic weapon data will be recommended to him to encourage the player to continue playing or paying. Among them, the target strategy data includes the above-mentioned strategy data and magic weapon data.

[0070] As the business continues to develop, the timeliness of user contact is becoming increasingly demanding, and there are more and more business scenarios that require real-time detection and contact with users, such as online birthday wishes, real-time push notifications for upgrades, real-time congratulations on sworn brotherhood, speech detection, etc. Therefore, the above-mentioned target policy data of this embodiment is policy data formulated for real-time contact with users, so that the target trigger condition can be a real-time trigger condition, which can also be called a real-time condition or a real-time trigger requirement.

[0071] As an optional implementation, the data processing method further includes the following steps:

[0072] Step 1: Store the target policy data in the second target database through the server.

[0073] In this embodiment, the front end can write the target policy data of the service configuration into the target policy table in the second target database for storage via the interface provided by the server. Figure 6 FIG. 1 is a schematic diagram of target strategy data according to an embodiment of the present invention. Figure 6 As shown, the target policy data is displayed in the graphical user interface, which may include policy set name, policy ID and policy name, creator, policy category, life cycle, creation date, release status, grayscale mechanism and progress, release volume and success volume, etc. In this embodiment, the target policy data may be a strategy formulated by some business that can be pushed when the user plays certain types of gameplay in real time in the game, for example, a strategy is pushed when a certain gameplay fails, and some magic weapons are recommended when a certain gameplay is played, so as to encourage the player to continue playing or pay.

[0074] Step 2: Send a detection request to the server.

[0075] In this embodiment, the target policy data includes target trigger conditions. The front end can send a detection request to the server, which is used to request the server to detect whether the behavior data of the target object meets the target trigger conditions stored in the second target database. Then, when the behavior data of at least one target object meets the target trigger conditions, the target push data corresponding to the target trigger conditions can be pushed to at least one target object.

[0076] In this embodiment, a Flink real-time strategy detection program (running in real time in the background) can be deployed on the data side to consume Kafka data in real time, mainly to detect in real time which target objects' behavior data meet the target trigger conditions set by which target strategy data.

[0077] Through the above steps S22 to S26 of the present application, the target tag and the target trigger condition corresponding to the target tag are determined; the target policy data is determined based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to at least one target object. In other words, this embodiment determines the target trigger condition based on the configured target tag, and then determines the target policy data based on the target trigger condition, thereby achieving the purpose of giving the business autonomous custom configuration of real-time trigger conditions, and real-time detection and reach of user behavior, solving the technical problem of the difficulty of formulating real-time reach strategies in the prior art, and achieving the technical effect of reducing the difficulty of formulating real-time reach strategies.

[0078] The above method of this embodiment is further introduced below.

[0079] As an optional implementation, the data processing method further includes: displaying configuration information of at least one target tag and / or a verification result of at least one target tag on a graphical user interface.

[0080] In this embodiment, the configuration information of the target tag may include but is not limited to: the tag name, sensitivity, data type, tag description, value caliber information, etc. of the target tag. The value caliber information may include: database, data table name, user account, role identifier (id), server, target tag value field (value), call deadline, etc. Figure 7 As shown. Among them, Figure 7 This is a target tag configuration information interface display diagram according to an embodiment of the present invention.

[0081] As another optional implementation, the data processing method further includes: determining that when the behavior data of at least one target object in the first target object set meets the target trigger condition, pushing target push data to the at least one target object.

[0082] In this embodiment, the first target object set can be determined according to the grouping function control on the graphical user interface. The first target object set, i.e., the user list, can include multiple users, such as user A, user B, user C, etc. In this embodiment, the first target object set can be loaded from the distributed file system (HDFS) file, and it can be determined whether there is a target object whose behavior data meets the target trigger condition in the first target object set. If so, the target push data is pushed to the target object. For example, when the behavior data of user A and user B in the user list meets the target trigger condition, the target push data is pushed to user A and user B.

[0083] As another optional implementation, the data processing method further includes: determining that when the behavior data of at least one target object in the second target object set meets the target trigger condition, pushing the target push data to the at least one target object is prohibited.

[0084] In this embodiment, a second target object set can be set according to the blacklist function control on the graphical user interface, and the objects in the second target object set are prohibited from receiving the target push data corresponding to the target trigger condition, that is, the second target object set is a blacklist. Optionally, this embodiment can load the second target object set from the HDFS file, and even when the behavior data of at least one target object in the second target object set meets the target trigger condition, it is prohibited to push the target push data to at least one target object. For example, there is a user D who is in the blacklist. Even if the behavior data of user D meets the target trigger condition, the target push data corresponding to the target trigger condition is not pushed to user D.

[0085] Optionally, this embodiment can read the behavior data of users in Kafka in real time, and can use program detection logic to define some operators, through which operators are used to detect which users' behavior data meet the target trigger conditions set by the target policy data, and then the list of target objects that meet the target trigger conditions in the target policy data can be output to the downstream in real time. Optionally, this embodiment can read the game log stream from the Kafka Topic of the corresponding game in real time based on Flink, and can load the configuration of the target policy data and the target trigger conditions configured in the target policy data from the configuration table of the target policy data. If user grouping or blacklist is selected, the corresponding first target object set or second target object set is loaded from the calculated HDFS file, and then it is determined that when the behavior data of at least one target object in the first target object set meets the target trigger condition, the target push data is pushed to at least one target object, or it is determined that when the behavior data of at least one target object in the second target object set meets the target trigger condition, it is prohibited to push the target push data to at least one target object.

[0086] Optionally, if this embodiment does not select a specific grouping or blacklist, this type of target policy data does not need to be effective for a specific set of target objects, and the real behavior data of any user will push the corresponding target data to the user as long as it meets the target trigger conditions.

[0087] As an optional implementation, the data processing method further includes: displaying target policy data and / or target push data.

[0088] In this embodiment, the business can customize target policy data on the graphical user interface, display target policy data on the graphical user interface, and display target push data to facilitate the business to view the target policy data and / or information related to the target policy data.

[0089] Figure 8 FIG. 1 is a schematic diagram of another target strategy data according to an embodiment of the present invention. Figure 8 As shown, when the business configures the strategy on the front-end page, you can choose to configure the real-time trigger conditions when defining the population. You can configure the grouping, blacklist, trigger conditions, maximum release volume and other data on the graphical user interface.

[0090] Fig. 9 is a schematic diagram of a Flink real-time detection program sample according to an embodiment of the present invention. Fig. 9 As shown, for the target strategy data that is running in real time in a certain game, the following steps can be performed:

[0091] Step S91, read the user's behavior data in Kafka in real time.

[0092] Step S92, using program detection logic to define some operators for detecting which users' behavior data meet the target triggering conditions set by the target policy data.

[0093] Step S93: outputting in real time a list of users that meet the target triggering conditions in the target policy data.

[0094] The above S93 of this embodiment is further introduced below.

[0095] In step 3.1, the Flink-based real-time strategy detection program reads the game log stream from the Kafka Topic of the corresponding game in real time.

[0096] Step 3.2, load the configuration of the target policy data and the target trigger condition configured in the target policy data from the policy configuration table.

[0097] Step 3.3: If the target policy data selects grouping or blacklist, the corresponding policy list, that is, the user list, is loaded from the calculated HDFS file.

[0098] In this embodiment, if the business does not select a specific group or blacklist at the front end, the target policy data of this type does not need to be effective for a specific policy list. As long as the real behavior data of any user meets the real-time trigger conditions, the corresponding content will be pushed to the user.

[0099] If the business selects grouping or blacklisting at the front end, this type of policy needs to be targeted at a specific policy list to take effect, or needs to be filtered out for a specific blacklist. This type of policy not only requires the user's behavior data to meet the set real-time trigger conditions, but also requires the user to be in the policy list or blacklist set by this policy data.

[0100] Step 3.4: In the strategy detection logic module, it is detected in real time whether the user's behavior meets the target triggering condition in the target strategy data.

[0101] This embodiment can write the target object that meets the target trigger condition into another Kafka topic. In the push service, the Kafka topic data can be consumed in real time and the corresponding target data can be pushed to the target object.

[0102] Compared with the existing Hive SQL solution, the timeliness of the data processing method proposed in the embodiment of the present application is more prominent. Taking the overall process of data transmission as an example, the timeliness of the Hive SQL solution and the data processing method proposed in the embodiment of the present application is compared.

[0103] In the related technology, the Hive SQL solution is a quasi-real-time solution. First, it needs to write the real-time user behavior data on the Kafka data to hdfs, and then add it to the corresponding partition of hive. The processing time of this step is at the hour level; secondly, the business personnel write SQL and obtain the user list that meets the requirements based on hive. The processing time of this step is at least at the minute level; finally, the user list is uploaded to the corresponding push system for push. This step also takes a certain amount of time to upload the list. In the Hive SQL solution, the transmission path of user data is: Kafka=>hive=>sql list=>push system, and user data is transmitted in multiple different systems. When applying the Hive SQL solution, for example, to detect user login: after a user logs in, it generally takes hours to push.

[0104] However, in the data processing method proposed in the embodiment of the present application, the policy configuration of the business can be stored in advance, wherein the trigger conditions are defined in the policy configuration. When the business completes the configuration at the front end, it will be stored in the database in real time. The data processing method proposed in the embodiment of the present application does not involve storing the real behavior data of the user, and thus can effectively save the data processing time. In the data processing method proposed in the embodiment of the present application, the data transmission path for real-time monitoring of users is: Kafka (real-time user behavior data) => real-time detection based on the Flink streaming computing framework => Kafka (list of users who meet the requirements) => real-time reading of push service. The entire solution, from the occurrence of the real user behavior to the push, usually only takes seconds and no more than one minute. Therefore, the data processing method proposed in the embodiment of the present application can achieve the purpose of giving the business autonomous custom configuration real-time trigger conditions, as well as real-time detection and reach of user behavior.

[0105] Fig.10 FIG. 1 is a schematic diagram of detecting real-time behavior of massive users based on Flink according to an embodiment of the present invention. Fig.10 As shown in the figure, the policy configuration parsing module in Flink can obtain the policy configuration table, the policy list loading module in Flink can obtain the policy list, and the real-time log loading module in Flink can obtain the real-time game log. The processing results of the policy configuration parsing module, the policy list loading module, and the implementation log loading module are input into the policy detection logic module to provide push services to users in the push list.

[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0107] The embodiment of the present invention further provides a data processing device. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "unit" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable. It should be noted that the data processing device of this embodiment can be used to execute the embodiments of the present invention. Figure 2 The data processing method shown.

[0108] Fig.11 is a schematic diagram of a data processing device according to an embodiment of the present invention. Fig.11 As shown, the data processing device 110 includes: a first determining unit 111 and a second determining unit 112 .

[0109] The first determining unit 111 is configured to determine a target tag and a target triggering condition corresponding to the target tag.

[0110] The second determination unit 112 is used to determine target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object.

[0111] Optionally, the first determining unit 111 includes: a first determining module, configured to determine a target tag based on the first log.

[0112] Optionally, the device also includes: a first storage unit, used to store the target tag to the first target database through the server; a first sending unit, used to send a verification request to the server, wherein the verification request is used to request the server to verify the target tag in the first target database, and if the number of second logs including the target tag is greater than or equal to the target threshold, the target tag is successfully verified by the server.

[0113] Optionally, the first determining unit 111 includes: a second determining module, configured to determine a target trigger condition based on a successfully verified target tag.

[0114] Optionally, the device further includes: a first display unit, configured to display configuration information of at least one target tag and / or a verification result of the target tag.

[0115] Optionally, the first determining unit 111 includes: a combining module, configured to, when there are multiple target tags, logically combine multiple target tags to obtain a target trigger condition.

[0116] Optionally, the device further includes: a pushing unit, configured to push target pushing data to the at least one target object when the behavior data of at least one target object in the first target object set meets the target trigger condition.

[0117] Optionally, the device further includes: a prohibition unit, configured to prohibit pushing target push data to at least one target object when the behavior data of at least one target object in the second target object set meets the target trigger condition.

[0118] Optionally, the device also includes: a second storage unit, used to store the target policy data to a second target database through a server; a second sending unit, used to send a detection request to the server, wherein the detection request is used to request the server to detect whether the behavior data meets the target trigger conditions stored in the second target database, wherein the target policy data includes the target trigger conditions.

[0119] Optionally, the device further includes: a second display unit, configured to display target policy data and / or target push data.

[0120] Through the above-mentioned embodiments of the present application, the target trigger conditions are determined based on the configured target tags, and then the target policy data is determined based on the target trigger conditions, thereby achieving the purpose of giving the business autonomous custom configuration of real-time trigger conditions and real-time detection and reach of user behavior, solving the technical problem of the difficulty of formulating real-time reach strategies in the prior art, and achieving the technical effect of reducing the difficulty of formulating real-time reach strategies.

[0121] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0122] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0123] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0124] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0125] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0126] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0127] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0132] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Determine a target tag and a target trigger condition corresponding to the target tag, wherein each target tag is used to represent information of a target field in the first log; Determining target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object; The method further includes: verifying whether there is a second log including the target tag in the log stream; if the number of the second logs is less than the target threshold, verifying the target tag based on the verification time of the target tag and the preset time; if the number of the second logs is greater than or equal to the target threshold, determining that the target tag verification is successful; If the number of the second logs is less than the target threshold, the target tag is verified based on the verification time and the preset time of the target tag, including: if the number of the second logs is less than the target threshold and the verification time is greater than or equal to the preset time, determining that the target tag verification has timed out; if the number of the second logs is less than the target threshold and the verification time is less than the preset time, returning to verify the second log.

2. The method according to claim 1, characterized in that: Determining the target tag includes: The target tag is determined based on the first log.

3. The method according to claim 2, characterized in that Determining the target tag based on the first log includes: Obtaining a target field in the first log; The value of the target field is determined as the target tag.

4. The method according to claim 2, characterized in that: The method further comprises: Storing the target tag in a first target database via a server; A verification request is sent to the server, wherein the verification request is used to request the server to verify the target tag in the first target database, and if the number of the second logs is greater than or equal to the target threshold, the target tag is successfully verified by the server.

5. The method according to claim 1, characterized in that Determining a target trigger condition corresponding to the target tag includes: The target trigger condition is determined based on the successfully verified target tag.

6. The method according to claim 1, characterized in that The method further comprises: The configuration information of the at least one target tag and / or the verification result of the target tag are displayed.

7. The method according to claim 1, characterized in that In the case that there are multiple target tags, determining a target trigger condition corresponding to the target tag includes: A plurality of the target tags are logically combined to obtain the target trigger condition.

8. The method according to claim 1, characterized in that: The method further comprises: It is determined that when the behavior data of the at least one target object in the first target object set meets the target trigger condition, the target push data is pushed to the at least one target object.

9. The method according to claim 1, characterized in that: The method further comprises: It is determined that when the behavior data of the at least one target object in the second target object set meets the target trigger condition, pushing the target push data to the at least one target object is prohibited.

10. The method according to claim 1, characterized in that The method further comprises: storing the target policy data in a second target database via a server; A detection request is sent to the server, wherein the detection request is used to request the server to detect whether the behavior data meets the target trigger condition stored in the second target database, wherein the target policy data includes the target trigger condition.

11. The method according to claim 1, characterized in that: The method further comprises: The target policy data and / or the target push data are displayed.

12. A data processing device, characterized in that: include: A first determining unit, configured to determine a target tag and a target trigger condition corresponding to the target tag, wherein each target tag is used to represent information of a target field in the first log; a second determining unit, configured to determine target policy data based on the target trigger condition, wherein the target policy data is used to indicate that when the behavior data of at least one target object meets the target trigger condition, the target push data corresponding to the target trigger condition is allowed to be pushed to the at least one target object; The device is further configured to perform the following steps: verifying whether there is a second log including the target tag in the log stream; if the number of the second logs is less than a target threshold, verifying the target tag based on the verification time of the target tag and a preset time; if the number of the second logs is greater than or equal to the target threshold, determining that the target tag verification is successful; Among them, the device is also used to execute the following steps to verify the target tag based on the verification time and preset time of the target tag if the number of the second logs is less than the target threshold: if the number of the second logs is less than the target threshold, and the verification time is greater than or equal to the preset time, determine that the target tag verification has timed out; if the number of the second logs is less than the target threshold, and the verification time is less than the preset time, return to verify the second log.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the method described in any one of claims 1 to 11.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the computer program to perform the method according to any one of claims 1 to 11.

Images