User Integral Processing Method, Device, Equipment, Medium and Program Product

Abstract

The present disclosure provides a user integral processing method. The method includes: calculating a first integral result based on static data, where the static data includes first user data generated according to a first behavior of the user, and the first behavior is used to indirectly obtain user integral; calculating a second integral result based on real-time data, where the real-time data includes second user data generated according to a second behavior of the user, and the second behavior is used to directly obtain user integral; respectively processing the first integral result and the second integral result to obtain N integral messages, and writing the N integral messages into a message queue; consuming the N integral messages from the message queue to calculate and obtain the user integral. The present disclosure also provides a user integral processing device, equipment, storage medium and program product.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and more particularly, to a user integral processing method, apparatus, device, medium, and program product. Background Art

[0002] Some companies can accumulate points based on user behaviors, enabling users to use the accumulated points as an exchange carrier for redeeming rights and interests, thereby achieving the purpose of rewarding users and enhancing user stickiness.

[0003] In the process of calculating user points, due to the complexity of user behavior scenarios and the fact that points are derived from different user behaviors. Therefore, the point processing end may be at the end of the company's data chain. For example, the point calculation process is handed over to the front end of the data chain, and only the calculated point results are used to update user points. Among them, the point processing end may also receive point results from multiple channels, and the data formats are diverse.

[0004] Therefore, in the related art, due to the connection between the front and back ends of the data chain, the data formats are diverse, the job tasks are interdependent, and the common logic cost is relatively high during the point processing process. Therefore, how to improve the flexibility, scalability, and point processing speed during the point processing process is an urgent problem to be solved currently. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides a user integral processing method, apparatus, device, medium, and program product that can improve the flexibility, scalability, and integral processing speed during the integral processing process.

[0006] In one aspect of the embodiments of the present disclosure, a user integral processing method is provided, including: calculating a first integral result based on static data, where the static data includes first user data generated according to a first behavior of a user, and the first behavior is used to indirectly obtain user points; calculating a second integral result based on real-time data, where the real-time data includes second user data generated according to a second behavior of the user, and the second behavior is used to directly obtain user points; respectively processing the first integral result and the second integral result to obtain N integral messages, and writing the N integral messages into a message queue, where N is an integer greater than or equal to 2; consuming the N integral messages from the message queue to calculate and obtain the user points.

[0007] According to an embodiment of the present disclosure, the static data includes integral files from S channels, and the calculation of the first integral result based on the static data includes: obtaining the integral files of the S channels, where the S channels include channels that provide services to the user in response to the first behavior and / or the second behavior, and the integral files include the first user data, where S is an integer greater than or equal to 1; preprocessing the integral files of each channel based on the preprocessing rules associated with each channel among the S channels.

[0008] According to an embodiment of the present disclosure, after preprocessing the integral files of each channel, a target table of each channel is obtained, and the target table includes M transaction records. The method further includes: matching each transaction record among the M transaction records with a corresponding integral calculation rule, where the M transaction records include the first user data, and M is an integer greater than or equal to 1; calculating the first user data in each transaction record based on the integral calculation rule to obtain the first integral result.

[0009] According to an embodiment of the present disclosure, the real-time data includes real-time request messages from S channels, and the calculation of the second integral result based on the real-time data includes: calling a corresponding first online service according to the channel to which the real-time request message belongs, where the real-time request message includes the second user data; calculating the integral of the second user data based on the first online service to obtain the second integral result.

[0010] According to an embodiment of the present disclosure, calculating the user integral by consuming the N integral messages from the message queue includes: calling a second online service based on each of the N integral messages; updating the user integral in the user integral account using the second online service according to the integral message.

[0011] According to an embodiment of the present disclosure, calling the second online service based on each of the N integral messages includes: converting each integral message into a corresponding online request; calling the second online service based on the online request, so that the second online service parses the online request to update the user integral account.

[0012] According to an embodiment of the present disclosure, the method further includes: writing a processing record in the index library, where the processing record includes a record of consuming integral messages from the message queue; before converting each integral message into a corresponding online request, the method further includes: querying the processing record of each integral message through the index library.

[0013] According to an embodiment of the present disclosure, the processing record includes a processing status. In the event that an abnormality occurs when the second online service parses the online request to update the user points account, the method further includes: modifying the processing status of the points message corresponding to the online request in the index library, which includes: modifying the processing status to an abnormal state.

[0014] Another aspect of the disclosed embodiment provides a user points processing device, including: a static data calculation module, a real-time data calculation module, a points result conversion module and a points message consumption module. The static data calculation module is used to obtain a first points result based on static data calculation, wherein the static data includes first user data generated according to a first behavior of the user, and the first behavior is used to indirectly obtain user points; the real-time data calculation module is used to obtain a second points result based on real-time data calculation, wherein the real-time data includes second user data generated according to a second behavior of the user, and the second behavior is used to directly obtain user points; the points result conversion module is used to process the first points result and the second points result respectively to obtain N points messages, and write the N points messages into a message queue, wherein N is an integer greater than or equal to 2; the points message consumption module is used to consume the N points messages from the message queue to calculate the user points.

[0015] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method as described above.

[0016] Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.

[0017] Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0018] One or more of the above embodiments have the following beneficial effects: Compared with the process of connecting the front and back ends of the data chain to process the points in the related art, by directly calculating the static data and the real-time data separately, and introducing the message queue to achieve the compatible processing of the static data and the real-time data, the first and second points results can be converted into points messages, and the processing of user points can be achieved through the consumption points message. Therefore, both static data and real-time data can be processed flexibly and in a targeted manner, and the front and back ends of the data chain are decoupled, which reduces the common logic cost and improves the points processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above content and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0020] Figure 1 Schematically shows the architecture diagram of a user integral processing system according to an embodiment of the present disclosure;

[0021] Figure 2 Schematically shows the flowchart of a user integral processing method according to an embodiment of the present disclosure;

[0022] Figure 3 Schematically shows the flowchart of obtaining the first integral result in operation S210 according to an embodiment of the present disclosure;

[0023] Figure 4 Schematically shows the interaction diagram of a static data source, a first cluster, and a second cluster according to an embodiment of the present disclosure;

[0024] Figure 5 Schematically shows the interaction diagram of a second cluster, a third cluster, and a rule engine according to an embodiment of the present disclosure;

[0025] Figure 6 Schematically shows the flowchart of obtaining the second integral result according to an embodiment of the present disclosure;

[0026] Figure 7 Schematically shows the interaction diagram of a real-time data source, a fourth cluster, and a message middleware according to an embodiment of the present disclosure;

[0027] Figure 8 Schematically shows the flowchart of obtaining user integral according to an embodiment of the present disclosure;

[0028] Figure 9 Schematically shows the flowchart of calling a second online service according to an embodiment of the present disclosure;

[0029] Figure 10 Schematically shows the interaction diagram among a message middleware, a fifth cluster, and a sixth cluster according to an embodiment of the present disclosure;

[0030] Figure 11 Schematically shows the structural block diagram of a user integral processing device according to an embodiment of the present disclosure;

[0031] Figure 12 Schematically shows the block diagram of an electronic device suitable for implementing the user integral processing method according to an embodiment of the present disclosure. Detailed implementation manners

[0032] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0033] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0034] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0035] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0036] In the technical solution of the present disclosure, the processing of data such as acquisition, collection, storage, use, processing, transmission, provision, disclosure, and application complies with the provisions of relevant laws and regulations, takes necessary confidentiality measures, and does not violate public order and good customs.

[0037] Taking a company that uses Internet technology to provide services as an example, users can log in to the company's client to perform operations such as consumption, check-in, and lottery. The client can record the data generated by various operations of the user and transmit it to the client server. The integral processing system can obtain the data from the client server and process the user's integral. Among them, the link through which the data generated by the user operation flows from the client to the backend server and then from the backend server to the integral processing system can be called a data chain. Since the client directly faces the user, the client is located at the front end of the data chain, while the integral processing system is located at the backend of the data chain.

[0038] In the related art, the client server can process the data of the user's operation to obtain the points result, and then push it to the points processing system. Due to the connection between the front and back ends of the data chain, the data format is diverse in the points processing process (connecting to different client servers), the tasks are interdependent (such as the task dependency in the client server, or the task dependency between the client server and the points processing system), the common logic cost is high, and the client server may push the batch points results to the points processing system at regular intervals, and the push time from different client servers is inconsistent, resulting in the points processing system being unable to update the user's points in a timely manner.

[0039] The embodiments of the present disclosure provide a method, apparatus, device, medium and program product for processing user points. The method includes: obtaining a first point result based on static data calculation, wherein the static data includes first user data generated according to a first behavior of the user, and the first behavior is used to indirectly obtain user points. Obtain a second point result based on real-time data calculation, wherein the real-time data includes second user data generated according to a second behavior of the user, and the second behavior is used to directly obtain user points. Process the first point result and the second point result separately to obtain N point messages, and write the N point messages to a message queue, wherein N is an integer greater than or equal to 2. Consume N point messages from the message queue to calculate and obtain user points.

[0040] Compared with the process of connecting the front and back ends of the data chain to process the points in the related art, the disclosed embodiment can directly obtain the source data from the front end of the data chain, directly calculate the static data and real-time data separately, and introduce the message queue to achieve compatible processing of static data and real-time data, and can convert the first and second points results into points messages, and realize the processing of user points through consumption points messages. Therefore, static data or real-time data can be processed flexibly and targetedly, and the front and back ends of the data chain are decoupled, which reduces the common logic cost and improves the speed of points processing.

[0041] Figure 1 The following schematically shows an architecture diagram of a user points processing system according to an embodiment of the present disclosure.

[0042] like Figure 1As shown in the figure, the user score processing system 100 according to this embodiment may include a data source 110, a first cluster 120, a second cluster 130, a third cluster 140, a rule engine 150, a fourth cluster 160, a message middleware 170, a fifth cluster 180, and a sixth cluster 190. Among them, the data source 110 may include a static data source 111 and a real-time data source 112. Among them, the first cluster 120, the second cluster 130, the third cluster 140, the fourth cluster 160, the fifth cluster 180, and the sixth cluster 190 may be distributed clusters respectively. The message middleware 170 may be a message cluster for providing message queue services.

[0043] The data source 110 is used to receive static data sources, such as first user data generated according to the first behavior of the user, and can also be used to receive real-time data sources, such as second user data generated according to the second behavior.

[0044] The first cluster 120 may include multiple server clusters. Among them, each server cluster may include several servers. The first cluster 120 may be used for preprocessing static data.

[0045] The second cluster 130 may include a Hadoop cluster, which is used to receive the preprocessed data from the first cluster 120 and further process and calculate the preprocessing. Among them, the Hadoop cluster is a platform suitable for distributed storage and distributed computing of massive data, and it may include a distributed storage framework HDFS component, a distributed computing framework MapReduce component, and a resource scheduling platform yarn component.

[0046] The third cluster 140 may include a Spark cluster, which is used to read the files processed by the second cluster 130 and perform concurrent processing item by item. Specifically, rule matching can be performed item by item, and score calculation can be performed according to the matched rules. The Spark cluster may be used as a caller for resource allocation. Among them, the Spark applications running in the Spark cluster run independently and are isolated from each other to a certain extent.

[0047] The rule engine 150 may be used to call one or more score rules, and in response to the call of the third cluster 140, screen the score rules and return the matched score rules.

[0048] The fourth cluster 160 may include multiple online application clusters for providing first online services.

[0049] The message middleware 170 may include a Kafka message cluster for providing a message queue service. When saving messages, the Kafka message cluster can classify them according to topics. Each topic can be divided into multiple partition groups, and each partition is an ordered queue. Each message in a partition will be assigned an ordered ID.

[0050] The fifth cluster 180 may include a streaming computing Storm cluster for listening to messages in the Kafka message cluster and performing streaming processing.

[0051] The sixth cluster 190 may include multiple online application clusters for providing a second online service.

[0052] The user points processing system 100 can be used to implement the user points processing method of the embodiments of the present disclosure, and can also be used to deploy the user points processing device of the embodiments of the present disclosure.

[0053] The following will be based on Figure 1 the described user points processing system, taking a financial institution's user points processing as an example, and will Figures 2 to 10 describe in detail the user points processing method of the embodiments of the present disclosure.

[0054] Figure 2 A flowchart of the user points processing method according to an embodiment of the present disclosure is schematically shown.

[0055] As Figure 2 shown, the user points processing method of this embodiment includes operations S210 to S240.

[0056] In operation S210, a first points result is calculated based on static data, where the static data includes first user data generated according to the first behavior of the user, and the first behavior is used to indirectly obtain user points.

[0057] The so-called indirect obtaining here means that the first behavior is not a behavior that directly obtains user points, and the points that can be obtained cannot be directly determined from the first user data. Taking a financial institution as a bank institution as an example, the first behavior may include, but is not limited to, behaviors such as using a credit card for consumption, using a debit card for consumption, and achieving the standard in participating in an activity. The first user data generated when the user uses a credit card for consumption may include credit card consumption statements, and the first user data generated when the user uses a debit card for consumption may include debit card consumption statements. For example, the first user data generated by the user's recommendation behavior when participating in a referral activity may include referral compliance records. Taking the referral activity as an example, if it is required to refer 10 people to allocate points, then when the user refers to the 5th person, this referral behavior cannot directly obtain points.

[0058] Since the first user data cannot directly determine the points, one or more users' first user data within a period of time can be obtained for batch processing. Therefore, it can be called static data. The first points result is, for example, calculating the first user data to determine the number of points that the user can obtain.

[0059] In operation S220, a second points result is obtained based on real-time data. Among them, the real-time data includes second user data generated according to the second behavior of the user, and the second behavior is used to directly obtain user points.

[0060] Here, directly obtaining, for example, means that the second behavior is directly associated with user points, and for the second user data, the points that can be obtained can be directly determined. Therefore, the second behavior includes behaviors that directly generate points in response to user operations, and the first behavior includes behaviors that generate points after further rule matching in response to user operations. For example, the second behavior includes, but is not limited to, the behavior of signing in to receive points on the bank client, the behavior of getting points for binding a card, and the behavior of getting points for consumption without a threshold. The second user data generated when the user signs in can include a sign-in record, the second user data generated when binding a card can include a card-binding record, and the second user data generated when consuming can include a consumption record.

[0061] Since the second behavior can directly obtain user points, the second user data can be processed in real time, that is, the points that can be obtained can be directly determined as the second user data is generated. Therefore, it can be called real-time data. The second points result is, for example, calculating the second user data to determine the number of points that the user can obtain.

[0062] In operation S230, the first points result and the second points result are respectively processed to obtain N points messages, and the N points messages are written into the message queue, where N is an integer greater than or equal to 2.

[0063] In operation S240, N points messages are consumed from the message queue to calculate and obtain user points.

[0064] See Figure 1 , the source file from which user data can be directly obtained is used for points calculation. On the one hand, the process of calculating points is no longer handed over to the front end of the data chain, so that the job tasks are no longer interdependent, and the front end of the data chain can directly push the generated source file. On the other hand, the user points processing system can calculate source files with different formats and different sources (such as different clients), and user points processing no longer depends on the push time of the front end, improving the processing speed.

[0065] Compared with the process of connecting the front and back ends of the data chain to process points in related technologies, by directly calculating static data and real-time data separately and introducing message queues to achieve compatible processing of static data and real-time data, the first and second points results can be converted into points messages, and the processing of user points can be achieved through consumption points messages. Therefore, static data or real-time data can be processed flexibly and in a targeted manner, and the front and back ends of the data chain are decoupled, which reduces the common logic cost and improves the speed of points processing.

[0066] Figure 3 The flowchart of obtaining the first integral result in operation S210 according to an embodiment of the present disclosure is schematically shown. Figure 4 The diagram schematically shows the interaction between a static data source, a first cluster and a second cluster according to an embodiment of the present disclosure. Figure 5 The diagram schematically shows the interaction between the second cluster, the third cluster and the rule engine according to an embodiment of the present disclosure.

[0067] like Figure 3 As shown, the user points processing method of this embodiment includes operations S310 to S340.

[0068] In operation S310, score files of S channels are obtained, wherein the S channels include channels that provide services to users in response to the first behavior and / or the second behavior, and the score files include first user data, wherein S is an integer greater than or equal to 1.

[0069] Channels include, but are not limited to, credit card channels, debit card channels, and target-reaching activity channels. For example, a credit card channel can provide credit card consumption services to users in response to credit card consumption behaviors of users. A debit card channel can provide debit card consumption services to users in response to debit card consumption behaviors of users. A target-reaching activity channel can provide users with services such as activity entrance and activity description in response to participation behaviors of users.

[0070] The points file may include the first user data of one or more users within a period of time. For example, a credit card points file may include the credit card consumption flow information of multiple users within a period of time. A debit card points file may include the debit card consumption flow information of multiple users within a period of time. A standard-reaching record points file may include the standard-reaching record information of multiple users within a period of time.

[0071] In operation S320, the score file of each channel is preprocessed based on the preprocessing rules associated with each channel in the S channels.

[0072] According to an embodiment of the present disclosure, the preprocessing rules can be implemented in the form of a configuration file by means of abstraction, centralization, and configuration. The configuration file may include preprocessing rules. First, for the characteristics of each of the S channels, the steps or objects to be processed are abstracted to form preprocessing rules adapted to each channel. Then, the configuration files of the S channels are centrally managed in a centralized manner, and each channel corresponds to at least one configuration file. Through the form of the configuration file, a configuration design of the preprocessing rules adapted to the S channels is realized. Thus, through operations on the configuration file, the dynamic change and expandable effect of the user processing logic can be achieved. It is possible to execute the preprocessing rules in the corresponding configuration file for the integral files of the S channels to process the user data in the integral files. Finally, the user points are obtained according to the processing results.

[0073] Referring to Figure 4 , the first cluster 120 can run batch applications 1, 2, 3... N. Each batch application can execute a preprocessing task to process batch data. For example, batch applications 1 to N can execute preprocessing tasks 1 to N one by one.

[0074] The first cluster 120 can determine the corresponding parsing configuration file based on the channel identifier of each channel; execute the parsing logic in the parsing configuration file to parse the integral file of each channel, where the parsing logic is used to convert the user data into user data in a predetermined format.

[0075] The parsing configuration file may include X parsing units, specifically including: determining X first target fields from the user data, where X is an integer greater than or equal to 1; mapping the values of the X first target fields to X columns in the first database table, where the X columns after mapping include user data in a predetermined format, and the X parsing units include X types of parsing logic corresponding to the X columns.

[0076] In some embodiments, the channel type numbers may be pre-agreed with S channels as channel identification identifiers. When batch-pushing the integral files into the first cluster 120, the file names are uniformly named according to the channel numbers. The integral files generated by each channel may have different formats, such as file formats, or different field meanings in the files. An alternative implementation is that each channel is docked with a fixed batch application, and executable scripts are stored in these batch applications. The integral files are parsed specifically by running the scripts. Among them, the batch application docked with the channel can be determined according to the file name of the integral file. Another alternative implementation is that each channel is not docked with a fixed batch application. After the first cluster 120 receives the integral file, it can be allocated to each batch application based on the load balancing algorithm. For example, after batch application 1 receives the integral file, it calls the associated executable script from the script library based on the file name to specifically parse the integral file. The process of running the script to parse the file is the process of running the preprocessing task, and the executable script includes preprocessing rules.

[0077] Since different channels may adopt different data processing processes, the contents and formats of the integral files received from each channel are different, and integral calculation cannot be performed through a unified calculation rule. Therefore, by using the preprocessing rules associated with each channel, the integral files can be preprocessed specifically, so as to obtain files in a unified format for unified integral calculation.

[0078] After preprocessing the integral files of each channel in operation S320, a target table for each channel is obtained, and the target table includes M flow records.

[0079] In operation S330, each flow record in the M flow records is matched with the corresponding integral calculation rule. The M flow records include first user data, where M is an integer greater than or equal to 1.

[0080] In operation S340, the first user data in each flow record is calculated based on the integral calculation rule to obtain the first integral result.

[0081] See Figure 4 and Figure 5 , Figure 5 Take the Hadoop cluster as the second cluster 130 and the Spark cluster as the third cluster 140 in

[0082] First, use the Hadoop cluster to split files. Leveraging the distributed file processing and computing capabilities of the Hadoop cluster, the pushed large file is split into files no larger than 50MB and stored distributively. Each Block in Hadoop can store massive amounts of data, meeting the requirements of large data volume scenarios and enabling fast access through the NameNode. When splitting the large file data using Mapreduce, the format is rewritten again to compensate for the default values, and duplicate data can be avoided from being loaded repeatedly. Finally, through task scheduling, the split files are output to the result directory according to a certain naming rule.

[0083] Second, use the Spark cluster to filter data. Spark reads the files split at the HDFS file path and processes them one by one concurrently. Data cleaning is performed according to pre-set data filtering rules, such as merchant filtering, blacklist filtering, amount filtering, etc. For merchant filtering, for example, merchants within a pre-determined industry can earn points. For blacklist filtering, for example, blacklisted merchants in a certain industry cannot earn points, or if a user is on the blacklist, they cannot earn points. For amount filtering, for example, if the amount is below a pre-determined amount, points cannot be earned. Spark calls the integrated rule Jar package layer by layer and completes the acquisition and authentication of user information online. Finally, the data that meets the rules is written into the shard files. Among them, after submitting a Spark command (such as a command to process a single transaction) on the Spark cluster client, a Drive thread is started. Figure 5 DriveA, DriveB, and DriveC can respectively represent different threads to handle merchant filtering, blacklist filtering, and amount filtering. Among them, DriveA1, DriveA2, DriveB1, DriveB2, DriveC1, and DriveC2 are child processes under the corresponding processes and are used to perform data filtering after matching at least one filtering rule. It should be noted that the filtering rules that can be matched in the Spark cluster are not limited to Figure 5 the merchant filtering, blacklist filtering, and amount filtering shown. It can be flexibly selected according to the actual situation.

[0084] The Spark cluster can determine the corresponding process configuration file based on the channel identifier of each channel; execute the process logic in the process configuration file to process the integrated files of each channel after parsing.

[0085] The process configuration file includes Y sets of process units, such as merchant filtering process unit set, blacklist filtering process unit set, and amount filtering process unit set. Executing the process logic in the process configuration file to process the integral file of each channel after parsing includes: sequentially executing each process unit set in the Y sets of process units based on the preset order of the Y sets of process units, where each process unit set corresponds to each processing process for obtaining the first integral result; wherein, each process unit set includes Z process units, and each process unit in the Z process units corresponds to each sub-processing process in each processing process, and each process unit includes at least one kind of process logic, and Y and Z are integers greater than or equal to 1 respectively.

[0086] Finally, use the Spark cluster for integral calculation. Spark reads the qualified data again, and calculates the integral value for the transaction records through the rules matched in the rule engine. For example, if the number of referrals of a user in a referral activity is 10, the integral value in the compliance record is calculated according to the configured rules. There may be a situation where one transaction record matches multiple integral rules, and multiple cumulative integral records (i.e., the first integral result) will be generated and written into the Kafka message cluster.

[0087] According to the embodiments of the present disclosure, the first behavior of a user may trigger different integral calculation rules or involve one or more activities. Integral matching is performed on the transaction records through the rule engine to improve the accuracy and efficiency of integral calculation.

[0088] Figure 6 Schematically shows a flowchart for obtaining the second integral result in operation S220 according to an embodiment of the present disclosure. Figure 7 Schematically shows an interaction diagram of a real-time data source, a fourth cluster, and a message middleware according to an embodiment of the present disclosure.

[0089] As Figure 6 shown, calculating the second integral result based on real-time data in operation S220 may include operations S610 to S620.

[0090] In operation S610, according to the channel to which the real-time request message belongs, call the corresponding first online service, where the real-time request message includes second user data.

[0091] In operation S620, perform integral calculation on the second user data based on the first online service to obtain the second integral result.

[0092] As Figure 7As shown, the fourth cluster 160 can receive real-time request messages from S channels. The fourth cluster 160 can include a router or gateway, A online applications 1, A online applications 2, A online applications 3... A online applications N, and a database. Among them, A online applications 1 to N can provide the first online service, and the first online service can perform fast storage processing of points in response to the received real-time request message to achieve instant response.

[0093] See Figure 1 and Figure 7 , first of all, the real-time data source can include data that is transmitted in real time by calling the API interface after the user's operation is received on the client side. For example, the real-time data can be transmitted to the fourth cluster 160 in the form of an HTTP message (i.e., a real-time request message) through an HTTP request.

[0094] Secondly, the fourth cluster 160 can use the router or gateway to distribute different HTTP messages to different A online applications. An optional implementation is that each channel corresponds to a fixed number of A online applications, and these A online applications correspond to the first online service. For example, the router can determine the channel to which it belongs based on the channel number in the HTTP message and then forward it to the corresponding A online application. Another optional implementation is that each channel does not correspond to a fixed number of A online applications, but corresponds to a fixed first online service. The router can distribute the HTTP message to an A online application based on the load balancing algorithm, and the A online application can determine the channel to which it belongs based on the channel number in the HTTP message and call the corresponding first online service.

[0095] Finally, use the first online service to parse the HTTP message to obtain the second user data and perform point calculation. Taking the example of a user participating in a lottery activity on the bank client, after the user performs a lottery operation, the lottery result can be obtained. The bank client can transmit the lottery result to the fourth cluster 160 in the form of an HTTP message in real time. The router in the fourth cluster 160 forwards the HTTP message to A online application 1. A online application 1 calls the first online service through the lottery channel number in the message to enable the first online service to obtain the lottery result. For example, if the lottery result is the third prize, the first online service can obtain the points that can be obtained for the third prize as the second point result.

[0096] In some embodiments, see Figure 7 , the second point result can be written into the database. And through the quartz automatic task mechanism, the second point result in the database is converted into a point message in an asynchronous manner and written into the message middleware 170.

[0097] Figure 8Schematically shows a flowchart of obtaining user points in operation S230 according to an embodiment of the present disclosure. Figure 10 Schematically shows an interaction diagram among a message middleware, a fifth cluster, and a sixth cluster according to an embodiment of the present disclosure.

[0098] As Figure 8 shown, consuming N point messages from a message queue in operation S230 to calculate and obtain user points may include operations S810 to S820.

[0099] In operation S810, call a second online service based on each of the N point messages.

[0100] As Figure 10 shown, the Storm cluster may be used as the fifth cluster 180. The sixth cluster 190 may receive requests from the Storm cluster. The sixth cluster 190 may include a router or gateway, B online applications 1, B online applications 2, B online applications 3... B online applications N, and a database. Among them, B online applications 1 to N may provide the second online service, and the second online service may update user points in units of user point accounts based on requests from the Storm cluster.

[0101] Referring to Figure 1 and Figure 10 , after being processed by the third cluster and the fourth cluster, the first point result and the second point result are converted into a point message data stream, and data processing is performed through a message middleware (such as a Kafka cluster) to bridge a streaming computing framework (the Storm cluster). In other words, the Storm cluster may act as a consumer to consume point messages from the message queue in the message middleware 170 to call the second online service.

[0102] In operation S820, update the user points in the user point account by using the second online service according to the point messages.

[0103] Before performing operation S820, at least one parameter identifier may be determined based on at least one parameter identifier in the third database table; at least one scenario field corresponding to the target parameter identifier may be determined from the fourth database table; the second target field may be matched with the at least one scenario field to obtain a matching result; and a corresponding configuration condition may be determined based on the matching result.

[0104] The third database table can be used as a parameter identification set definition table to define parameter identifications (such as names, numbers, etc.) and types. The fourth database table can be used as a parameter identification specific value table to define the enumerated values of parameter identifications, that is, scenario fields. Parameter identifications can refer to configuring the integral calculation rules in a parameterized manner. Each integral calculation rule can be assigned a corresponding parameter identification. Scenario fields can refer to the detailed scenario information specifically involved in user data, such as fields corresponding to several integral activities, integral stars, consumption amounts, merchant names, etc. It should be noted that the above integral calculation rules are different from the rule configuration forms called by the rule engine, and the content of the user data processed is also different.

[0105] Refer to Figure 10 , the router or gateway can perform load balancing on the received requests. For example, it can send the requests corresponding to an integral message to B Online Application 1. This integral message can include user information, cumulative integral values, etc. B Online Application 1 can call the second online service to obtain the user's integral account and query the integral balance (i.e., the user's integral) in the database. An optional implementation is that the second online service can add the cumulative integral value to the integral balance, and then determine whether it has exceeded the maximum allowed integral limit of the user's integral account. If it has exceeded, the integral balance is updated to the maximum limit. If it has not exceeded, it is directly updated to the added result. Finally, the updated user's integral account and the integral balance are written into the database. In some embodiments, the integral message can include user information, reduced integral values, etc.

[0106] On the one hand, after obtaining the first integral result based on static data, updating the integral to the user's integral account by parsing the message corresponding to the first integral result through the second online service can avoid the update logic being distributed in both online and batch locations, increasing the subsequent maintenance complexity. On the other hand, in the process of obtaining the second integral result based on real-time data, the first online service is called, and the second online service is called during the update. Through the combination of the first online service and the second online service, the instant processing of real-time data is realized, and the compatible processing of real-time data and static data is realized. In addition, adding scheduling control logic for integral account update on the second online service side can complete special business processing such as exception retry and unified management of large amounts of integrals.

[0107] Figure 9 Schematically shows a flowchart of calling the second online service in operation S810 according to an embodiment of the present disclosure.

[0108] As Figure 9 shown, calling the second online service for each of the N integral messages in operation S810 can include operations S910 to S920.

[0109] In operation S910, each integral message is converted into a corresponding online request.

[0110] The conversion process can be data format conversion, or valid data can be extracted from each integral message and then processed to obtain an online request.

[0111] In operation S920, a second online service is invoked based on the online request, so that the second online service parses the online request to update the user's integral account.

[0112] Refer to Figure 10 , the spout process of the Storm cluster can be used to listen for changes in the messages in the consumption queue, consume the integral messages and transfer them to the bolt thread group for dynamic balanced consumption. Write the bolt processing logic to convert the integral messages into HTTP online requests, and call the online service to complete the adjustment of the integral account. Specifically, in response to the topic classification in the message middleware, the integral messages can be consumed in the Storm cluster according to the topic, such as consuming topic1, consuming topic2, etc. Among them, consuming topic1 is used to process the integral messages from the credit card channel. Consuming topic2 is used to process the integral messages from the debit card channel. It should be noted that the user integral processing method of the embodiments of the present disclosure can set several channels as needed, and correspondingly, several topics can be set in the Storm cluster.

[0113] According to an embodiment of the present disclosure, a processing record can be written in the index library, where the processing record includes the record of consuming integral messages from the message queue. Before performing operation S910 to convert each integral message into a corresponding online request, it further includes: querying the processing record of each integral message through the index library.

[0114] As Figure 10 shown, it can be set that the index library 1010 is docked with the Storm cluster. The index library 1010 can store the records of integral messages processed by the Storm cluster in history. Before an integral message is transferred to the bolt thread group for dynamic balanced consumption, it can first query whether there is a processing record of this integral message in the index library 1010. If so, it will no longer continue to process. Its function is to prevent any link before consuming the integral message from repeatedly sending the processed data, thus avoiding the waste of computing resources caused by processing duplicate data and even the repeated accumulation of user points.

[0115] According to an embodiment of the present disclosure, the processing record includes a processing status. In the case where an exception occurs when the second online service parses the online request to update the user's integral account, it further includes: modifying the processing status of the integral message corresponding to the online request in the index library, where it includes: modifying the processing status to an abnormal status.

[0116] Reference Figure 10 Figure 10 , after the sixth cluster 190 receives an online request, a parsing failure may occur. In this case, the integral message corresponding to the online request can be sent back to the Sorm cluster, and the processing status of this integral message in the index library can be modified to an abnormal status. It is also possible that the update of the user's integral fails. In this case, the integral message corresponding to the online request can also be sent back to the Sorm cluster and the status can be modified. In some embodiments, an abnormal database can be configured for the sixth cluster 190 to store the online requests that have exceptions. Then, data synchronization can be performed between the abnormal database and the index library at regular intervals to update the status of the corresponding integral message.

[0117] By modifying the processing status of the integral message, duplicate prevention processing and compensation for abnormal data can be achieved. For example, before performing operation S910, it can be queried whether there is a processing record of the current integral message in the index library. If there is no record, the processing can continue. If there is a record and the processing status is a successful status, the processing will not continue. If there is a record and the processing status is an abnormal status, the processing can continue. Thus, a compensation mechanism is carried out after the second online service generates abnormal data to improve the correctness of user integral processing.

[0118] Based on the above user integral processing method, the present disclosure also provides a user integral processing device. The following will be combined with Figure 11 to describe this device in detail.

[0119] Figure 11 A structural block diagram of a user integral processing device 1100 according to an embodiment of the present disclosure is schematically shown.

[0120] As Figure 11 shown, the user integral processing device 1100 of this embodiment includes a static data calculation module 1110, a real-time data calculation module 1120, an integral result conversion module 1130, and an integral message consumption module 1140.

[0121] The static data calculation module 1110 can, for example, execute operation S210 to obtain a first integral result based on static data calculation. Among them, the static data includes first user data generated according to the first behavior of the user, and the first behavior is used to indirectly obtain the user's integral.

[0122] In some embodiments, the static data calculation module 1110 can, for example, also execute operations S310 to S320, which will not be elaborated here.

[0123] The real-time data calculation module 1120 can, for example, execute operation S220 to obtain a second integration result based on real-time data calculation. The real-time data includes second user data generated according to the second behavior of the user, and the second behavior is used to directly obtain user points.

[0124] In some embodiments, the real-time data calculation module 1120 can, for example, also execute operations S610 to S620, which will not be elaborated here.

[0125] The integration result conversion module 1130 can, for example, execute operation S230 to separately process the first integration result and the second integration result to obtain N integration messages, and write the N integration messages into a message queue, where N is an integer greater than or equal to 2.

[0126] In some embodiments, the integration result conversion module 1130 can, for example, also execute operations S810 to S820, and operations S910 to S920 will not be elaborated here.

[0127] The integration message consumption module 1140 can, for example, execute operation S240 to consume N integration messages from the message queue to calculate and obtain user points.

[0128] According to an embodiment of the present disclosure, any plurality of modules among the static data calculation module 1110, the real-time data calculation module 1120, the integration result conversion module 1130, and the integration message consumption module 1140 can be combined and implemented in one module, or any one of them can be split into multiple modules. Or, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the static data calculation module 1110, the real-time data calculation module 1120, the integration result conversion module 1130, and the integration message consumption module 1140 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in any other reasonable way such as hardware or firmware by integrating or packaging circuits, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Or, at least one of the static data calculation module 1110, the real-time data calculation module 1120, the integration result conversion module 1130, and the integration message consumption module 1140 can be at least partially implemented as a computer program module, and when the computer program module runs, it can execute corresponding functions.

[0129] Figure 12A block diagram of an electronic device suitable for implementing a user point processing method according to an embodiment of the present disclosure is schematically shown.

[0130] As Figure 12 shown, the electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage section 1208 into a random access memory (RAM) 1203. The processor 1201 can include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 can also include on-board memory for caching purposes. The processor 1201 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0131] In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are stored. The processor 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. The processor 1201 performs various operations of the method flow according to an embodiment of the present disclosure by executing the programs in the ROM 1202 and / or the RAM 1203. It should be noted that the programs can also be stored in one or more memories other than the ROM 1202 and the RAM 1203. The processor 1201 can also perform various operations of the method flow according to an embodiment of the present disclosure by executing the programs stored in one or more memories.

[0132] According to an embodiment of the present disclosure, the electronic device 1200 can further include an input / output (I / O) interface 1205, and the input / output (I / O) interface 1205 is also connected to the bus 1204. The electronic device 1200 can further include one or more of the following components connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc. An output section 1207 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. A storage section 1208 including a hard disk, etc. And a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read from it can be installed into the storage section 1208 as needed.

[0133] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist alone without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the methods according to the embodiments of the present disclosure are implemented.

[0134] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 1202 and / or RAM 1203 and / or one or more memories other than ROM 1202 and RAM 1203.

[0135] Embodiments of the present disclosure also include a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to cause the computer system to implement the item recommendation method provided by the embodiments of the present disclosure.

[0136] When the computer program is executed by the processor 1201, the above functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0137] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and is downloaded and installed through the communication part 1209, and / or installed from the removable medium 1211. The program code included in the computer program may be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to the embodiments of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0139] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure can be combined and combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0142] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are separately described above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A method for processing user points, comprising: Obtaining M transaction records for each of S channels, where the M transaction records include first user data generated based on a first behavior of the user, and the first behavior is used to indirectly obtain user points. Among them, the S channels include channels that provide services to the user in response to the first behavior and / or a second behavior, and M and S are integers greater than or equal to 1; Calculating the first user data in each transaction record based on the point calculation rules matching each transaction record of each channel to obtain a first point result; Invoking a corresponding first online service according to the channel to which the real-time request message from the S channels belongs, where the real-time request message includes second user data generated based on the second behavior of the user, and the second behavior is used to directly obtain user points; Calculating points for the second user data based on the first online service to obtain a second point result; Processing the first point result and the second point result respectively to obtain N point messages, and writing the N point messages into a message queue, where N is an integer greater than or equal to 2; Consuming the N point messages from the message queue to calculate and obtain the user points.

2. The method according to claim 1, wherein, The obtaining M transaction records for each of S channels includes: Obtaining the point files of the S channels, where the point files include the first user data; Preprocessing the point files of each channel based on the preprocessing rules associated with each of the S channels.

3. The method according to claim 2, wherein, After preprocessing the point files of each channel, obtaining a target table for each channel, where the target table includes the M transaction records. The method further includes: Matching a corresponding point calculation rule for each of the M transaction records, and the M transaction records include the first user data.

4. The method according to claim 1, wherein, The consuming the N point messages from the message queue to calculate and obtain the user points includes: Invoking a second online service based on each of the N point messages; Updating the user points in the user point account using the second online service according to the point message.

5. The method according to claim 4, wherein The invoking a second online service based on each of the N point messages includes: Converting each point message into a corresponding online request; Invoking the second online service based on the online request, so that the second online service parses the online request to update the user point account.

6. The method according to claim 5, wherein The method further includes: Writing a processing record in an index library, where the processing record includes a record of consuming point messages from the message queue; Before converting each point message into a corresponding online request, the method further includes: Querying the processing record of each point message through the index library.

7. The method according to claim 6, wherein, The processing record includes a processing status. In the case where an exception occurs when the second online service parses the online request to update the user point account, the method further includes: Modify the processing status of the integral message corresponding to the online request in the index library, including: modifying the processing status to an abnormal status.

8. A user integral processing device, comprising: A static data calculation module, configured to obtain M transaction records for each of the S channels, where the M transaction records include first user data generated based on the first behavior of the user, and the first behavior is used to indirectly obtain user integral. The S channels include channels that provide services to the user in response to the first behavior and / or the second behavior, and M and S are integers greater than or equal to 1; calculate the first user data in each transaction record based on the integral calculation rule matching each transaction record of each channel to obtain a first integral result; A real-time data calculation module, configured to call a corresponding first online service according to the channel to which the real-time request message from the S channels belongs. The real-time request message includes second user data generated based on the second behavior of the user, and the second behavior is used to directly obtain user integral; perform integral calculation on the second user data based on the first online service to obtain a second integral result; An integral result conversion module, configured to process the first integral result and the second integral result respectively to obtain N integral messages, and write the N integral messages into a message queue, where N is an integer greater than or equal to 2; An integral message consumption module, configured to consume the N integral messages from the message queue to calculate and obtain the user integral.

9. An electronic device, comprising: One or more processors; A storage device, configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images