A real-time label processing method and device, electronic equipment and storage medium

Abstract

The application provides a real-time label processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: monitoring a current user behavior that causes a change in a user state; obtaining a configuration relationship between the current user behavior and a data source; extracting data of a user real-time label from the corresponding data source according to the configuration relationship; and processing the data of the user real-time label to generate a new real-time label. The method at least solves the problem of poor real-time performance of a real-time label in the related art, thereby affecting the accuracy of the real-time label. The method is suitable for data processing, label processing and other scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a real-time label processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] According to the characteristic labels on the bodies of users and the like, the users can be screened and classified, and combined retrieval can be performed thereon to achieve accurate marketing. Among them, the account period label calculation is mostly based on days and months as the account period, such as a user's daily call volume as a standard daily account period label and monthly call volume as a standard monthly account period label. There is also a quasi-real-time calculation, which is a real-time label. Compared with the daily and monthly account period labels, the real-time label requires greater calculation amount and higher calculation speed.

[0003] The traditional real-time label calculation has the following problems: when the business system wants to use the real-time label, the real-time label is processed and extracted again. Due to the need for a large amount of real-time calculation, the speed of real-time label processing and extraction is slow, and the extraction often times out, resulting in poor real-time performance of the label, thereby affecting the accuracy of the real-time label. SUMMARY

[0004] The present application aims to solve the above problems in the prior art and provides a real-time label processing method and device, electronic equipment and a storage medium to at least solve the problem of poor real-time performance of the real-time label, thereby affecting the accuracy of the real-time label.

[0005] In a first aspect, the present application provides a real-time label processing method, comprising: monitoring a current user behavior that causes a user state to change; obtaining a configuration relationship between the current user behavior and a data source; extracting user real-time label data from the corresponding data source according to the configuration relationship; and processing the user real-time label data to generate a new real-time label.

[0006] Preferably, the obtaining of the configuration relationship between the current user behavior and the data source specifically comprises: obtaining the configuration relationship between the current user behavior and the data source from a preset database, wherein a distributed relational database stores user behaviors, data sources, triggered user real-time labels, structured query language (SQL) used by the user real-time labels, and mapping relationships therebetween.

[0007] Preferably, the number of user real-time labels is one or more.

[0008] The extracting of the user real-time label data from the corresponding data source according to the configuration relationship specifically comprises: executing an SQL statement to extract the user real-time label data from the corresponding data source according to the configuration relationship.

[0009] Preferably, the data of the user real-time label is processed to generate a new real-time label, specifically including: merging the data of the user real-time label into one piece of data; and performing data cleaning and processing on the merged data to generate a new real-time label.

[0010] Preferably, after the data of the user real-time label is processed to generate a new real-time label, the method further includes: storing the new real-time label in a database supporting high-concurrency queries in real time.

[0011] Preferably, after the new real-time label is stored in the database supporting high-concurrency queries in real time, the method further includes: receiving a query request for a user real-time label; and obtaining the user real-time label from the database supporting high-concurrency queries according to the query request.

[0012] In a second aspect, the present application further provides a real-time label processing device, including:

[0013] A monitoring module is configured to monitor a current user behavior leading to a change in a user state.

[0014] An obtaining module is connected to the monitoring module and configured to obtain a configuration relationship between the current user behavior and a data source.

[0015] An extracting module is connected to the obtaining module and configured to extract data of a user real-time label from a corresponding data source according to the configuration relationship.

[0016] A processing module is connected to the extracting module and configured to process the data of the user real-time label to generate a new real-time label.

[0017] Preferably, the device further includes a storage module.

[0018] The storage module is connected to the processing module and configured to store the new real-time label in a database supporting high-concurrency queries in real time.

[0019] In a third aspect, the present application further provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the real-time label processing method according to the first aspect.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the real-time label processing method according to the first aspect.

[0021] The application provides a real-time label processing method, device, electronic equipment and storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flowchart of a real-time label processing method according to Embodiment 1 of the application;

[0023] Figure 2 A flowchart of a real-time label processing method according to Embodiment 2 of the application;

[0024] Figure 3 A real-time label query process according to Embodiment 2 of the application;

[0025] Figure 4 A structure diagram of a real-time label processing device according to Embodiment 3 of the application;

[0026] Figure 5 A structure diagram of an electronic equipment according to Embodiment 4 of the application. DETAILED DESCRIPTION

[0027] To make the skilled in the art better understand the technical solutions of the application, the embodiments of the application will be further described in detail below with reference to the drawings.

[0028] It can be understood that the specific embodiments and drawings described herein are only used to explain the application, but not to limit the application.

[0029] It can be understood that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0030] It can be understood that, for the convenience of description, only the parts related to the application are shown in the drawings of the application, and the parts unrelated to the application are not shown in the drawings.

[0031] It can be understood that each unit and module involved in the embodiments of the application can only correspond to one entity structure, or can be composed of multiple entity structures, or multiple units and modules can be integrated into one entity structure.

[0032] It can be understood that the functions and steps marked in the flowcharts and block diagrams of the application can occur in an order different from that marked in the drawings without conflict.

[0033] It is understood that the flowcharts and block diagrams of this invention illustrate the possible architecture, functions, and operations of systems, apparatuses, devices, and methods according to various embodiments of this invention. Each block in the flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagram and flowchart can be implemented using a hardware-based system to achieve the specified function, or using a combination of hardware and computer instructions.

[0034] It is understood that the units and modules involved in the embodiments of the present invention can be implemented by software or by hardware. For example, the units and modules can be located in a processor.

[0035] Example 1:

[0036] like Figure 1 As shown, this embodiment provides a method for processing real-time tags, including:

[0037] Step 101: Monitor the current user behavior that causes a change in the user's state.

[0038] In this embodiment, user behavior includes actions such as account cancellation, service suspension, and single-stop service; internet behaviors such as visiting a website or using an application; location changes such as roaming outside the province or abroad; and warning behaviors such as sudden increases in data usage or call charges. Changes in some user behaviors lead to changes in the corresponding user status, which is described by user tags. By monitoring changes in user behavior in real time and generating new real-time user tags promptly, the system ensures that the tag data is up-to-date when querying user real-time tags, guaranteeing real-time performance.

[0039] Step 102: Obtain the configuration relationship between the current user behavior and the data source.

[0040] Specifically, obtaining the configuration relationship between the current user behavior and the data source includes: obtaining the configuration relationship between the current user behavior and the data source from a preset database, wherein the distributed relational database stores user behavior, data source, triggered real-time user tags, the structured query language SQL used by the real-time user tags, and the mapping relationship between the four.

[0041] In this embodiment, the database can be a distributed relational database. For example, when it is monitored that the user behavior and the user state change, the current user behavior and the configuration relationship of the data source are directly retrieved in the distributed relational database DRDS. The configuration relationship describes the following content: which user real-time tags are generated for each user behavior; and which SQL is used for processing each user real-time tag. For example, a user makes an action of "going abroad" (action code GOAP) at 09:02:34 on August 4, 2022. In the DRDS, it is stored that the business action "going abroad" triggers the change of 3 tags, that is, the user goes out of the country (GOAP-F), the user goes into the country (GOAP-T), and the user roaming time (GOAP-Time), as shown in Table 1 configuration relationship example:

[0042] Table 1 configuration relationship table

[0043] Sequence number User action code Trigger real-time tag Data source Extract SQL 1 GOAP GOAP-F U Select xxx from yyy; 2 GOAP GOAP-T P Select xxx from yyy; 3 GOAP GOAP-Time P Select xxx from yyy;

[0044] In Table 1, GOAP represents the roaming action of the user, and the action triggers the update of 3 real-time tags: GOAP-F represents going out of the country, GOAP-T represents going into the country, and GOAP-Time represents the user roaming time. U in the data source represents extracting the tag from the "China Unicom Group Company data source", and P represents extracting the tag from the "China Unicom provincial company business data source". The extraction SQL refers to the SQL statement executed in each data source, that is, the data of the real-time tag can be queried through the corresponding SQL.

[0045] In step 103, the data of the user real-time tag is extracted from the corresponding data source according to the configuration relationship.

[0046] In this embodiment, one or more tags exist on one user at the same time, and one user action can cause multiple tags to change at the same time. For example, after paying 100 yuan, the data of multiple tags such as "balance" and "monthly payment amount" can change at the same time.

[0047] Optionally, the number of user real-time tags is one or more.

[0048] Specifically, the extracting the data of the user real-time tag from the corresponding data source according to the configuration relationship includes: executing an SQL statement in the corresponding data source to extract the data of the user real-time tag according to the configuration relationship.

[0049] In this embodiment, after the payment action occurs, the corresponding SQL statements are executed in the corresponding data sources to extract the data of the "balance" and "monthly payment amount" tags respectively.

[0050] In step 104, the data of the user real-time tag is processed to generate a new real-time tag.

[0051] In this embodiment, the data processing of the user real-time label includes data cleaning and data processing:

[0052] (1) Label null replacement, such as using default value replacement, average value replacement, expert experience value replacement, etc., and the replacement rule is also stored in DRDS. For each real-time label, its replacement rule can be preset in advance.

[0053] (2) Label abnormal value replacement, for example, after executing SQL, the string 0 is taken out for the country, but 0 is obviously not a country. For such obvious value abnormality, the string 0 can be replaced with "China".

[0054] Optionally, the data of the user real-time label is processed to generate a new real-time label, specifically including: merging the data of the user real-time label into one data, and performing data cleaning and processing on the merged data to generate a new real-time label.

[0055] In this embodiment, the data transmitted to the business system should be a current change of one data, and simultaneously contain multiple real-time labels. After the data of the extracted user real-time label is summarized as one data, the new real-time label is processed and transmitted to the business system, which can avoid the waste of network communication resources caused by the feedback of real-time labels one by one.

[0056] Optionally, after the data of the user real-time label is processed to generate a new real-time label, the method further includes: storing the new real-time label in a database supporting high-concurrency query in real time.

[0057] In this embodiment, since the real-time label of the prior art completely depends on real-time calculation without a storage process of real-time label, when the front-end business system queries the same label concurrently, a large amount of meaningless repeated calculation will occur. Therefore, the new real-time label generated in this embodiment is written into hbase in the form of flink. Flink has the characteristics of fast writing speed, running in all common cluster environments, and calculating at memory speed and any scale, and the effect of batch processing is significant. Since the label is frequently queried with high concurrency, and hbase has the characteristics of high reliability, high performance, and scalable distribution, the new real-time label is stored in a database supporting high-concurrency query to avoid a large amount of repeated calculation and ensure the stability of the system.

[0058] Optionally, after the new real-time label is stored in the database supporting high-concurrency query in real time, the method further includes: receiving a query request of a user real-time label; and acquiring the user real-time label in the database supporting high-concurrency query according to the query request.

[0059] In this embodiment, if the business system needs to obtain the real-time label of the user, the real-time label of the user is directly obtained in the database supporting high-concurrency query according to the query request of the business system, and the obtained real-time label of the user is fed back to the business system through an interface, so that the query efficiency of the real-time label is greatly improved, the data storage safety is ensured, the concurrency of the query can be controlled, and system paralysis, query timeout and the like caused by high-concurrency access of the business system are avoided. Alternatively, the business system directly queries the real-time label of the user in the database supporting high-concurrency query, so as to improve the query efficiency.

[0060] The processing method of the real-time label in this embodiment, when the current user behavior causing the change of the user state is monitored, the configuration relationship between the current user behavior and the data source is obtained, and the data of the real-time label of the user is extracted from the corresponding data source according to the configuration relationship for processing, so as to generate a new real-time label. Since the real-time monitoring and real-time generation of the user label are performed, when the business system needs to use the real-time label, the processing and extraction of the real-time label are not needed, so that the real-time label has high real-time performance and ensures the accuracy of the real-time label. Secondly, after the extracted data of the real-time label of the user is summarized as one piece of data, the new real-time label is processed and transmitted to the business system, so that the network communication resources can be saved. Thirdly, the new real-time label is generated in the flink mode and written into the hbase in batches, so as to avoid a large amount of repeated calculation and ensure the stability of the system. Further, the real-time label of the user is directly obtained in the database supporting high-concurrency query according to the query request of the business system, and the obtained real-time label of the user is fed back to the business system through an interface, so that the query efficiency of the real-time label is greatly improved, the data storage safety is ensured, the concurrency of the query can be controlled, and the reliability of the system is ensured.

[0061] Embodiment 2

[0062] As shown in Figure 2 The embodiment provides a processing method of a real-time label, which is used for improving the processing speed of the real-time label, and the method comprises the following steps:

[0063] S1, in a business line module, monitoring the change of the behavior of a user; the user behavior action is a series of operation behaviors of the user, including but not limited to the following actions: user account cancellation, shutdown, single stop and various state change behaviors; user access to a website, use of an app and the like internet behaviors; displacement behaviors such as user location movement, roaming out of a province, roaming out of a country and the like; early warning behaviors such as user traffic and call charge surge.

[0064] S2, after monitoring the user behavior and state change, directly retrieve the current user behavior and the corresponding configuration relationship of the data source (this corresponding configuration relationship is configured in the distributed database DRDS); the configuration relationship describes two parts of content, one is that each user behavior will generate which user real-time tags, and the other is to describe what kind of SQL is used for processing each user real-time tag; for example, the user made the action of going abroad (action code GOAP) at 09:02:34 on August 4, 2022, and the business action of going abroad is stored in DRDS, which stores the change of 3 tags triggered by the action, i.e., user country out (GOAP-F), user country in (GOAP-T), and user roaming time (GOAP-Time), and the corresponding configuration relationship is as shown in the table 1 example of embodiment 1. Among them, DRDS is only a storage medium for configuration, and can also be other databases such as mysql, sqlserver, etc.

[0065] S3, after obtaining the corresponding relationship between each user behavior and the data source, execute the SQL statement in each data source to extract the data of each user tag. In the above “roaming” example, we extract 3 user tags, namely user country out (GOAP-F), user country in (GOAP-T), and user roaming time (GOAP-Time).

[0066] S4, after obtaining the above 3 tags of the two data sources, combine the 3 user tag data into one data, and prepare for data cleaning and processing.

[0067] S5, the data cleaning and processing process handles the following cases of real-time tags: label null value replacement (such as using default value replacement, average value replacement, expert experience value replacement, etc.; the replacement rule is also stored in DRDS, and the replacement rule is preset for each real-time tag); label abnormal value replacement, for example, after executing SQL, the country out is a string 0, but 0 is obviously not a country, which is a obvious value exception, and the default processing method is to replace it with “China”.

[0068] S6, use the flink method to batch write the generated real-time tags into hbase.

[0069] S7, when other systems want to obtain the real-time tags of a user, as shown in Figure 3 , indirectly query hbase through the interface or directly query hbase, without the need to calculate real-time tags when querying, which greatly improves the query efficiency.

[0070] The processing method of the real-time label of the embodiment simplifies the query process and improves the query efficiency by monitoring the user behavior and calculating the current multiple real-time labels of the user in real time. In addition, the real-time labels of the user are stored in the hbase in real time, so that other systems can obtain the user labels in real time.

[0071] Embodiment 3

[0072] As shown in Figure 4 The embodiment provides a real-time label processing device, which comprises:

[0073] The monitoring module 41 is configured to monitor the current user behavior that causes the change of the user state.

[0074] The acquisition module 42 is connected with the monitoring module 41 and is configured to acquire the configuration relationship between the current user behavior and the data source.

[0075] The extraction module 43 is connected with the acquisition module 42 and is configured to extract the data of the real-time label of the user from the corresponding data source according to the configuration relationship.

[0076] The processing module 44 is connected with the extraction module 43 and is configured to process the data of the real-time label of the user to generate a new real-time label.

[0077] Optionally, the acquisition module is configured to acquire the configuration relationship between the current user behavior and the data source from a preset database, wherein the distributed relational database stores the user behavior, the data source, the triggered real-time label of the user, the structured query language (SQL) used by the real-time label of the user, and the mapping relationship among the four.

[0078] Optionally, the number of the real-time labels of the user is one or more. The extraction module is configured to execute the SQL statement to extract the data of the real-time label of the user from the corresponding data source according to the configuration relationship.

[0079] Optionally, the processing module is configured to merge the data of the real-time label of the user into one piece of data, and perform data cleaning and processing on the merged data to generate a new real-time label.

[0080] Optionally, the device further comprises a storage module.

[0081] The storage module is connected with the processing module and is configured to store the new real-time label in a database supporting high-concurrency query in real time.

[0082] Optionally, the device further comprises a receiving module.

[0083] The receiving module is connected with the storage module, and is configured to receive a query request of a real-time label of a user, and acquire the real-time label of the user in a database supporting high-concurrency query according to the query request.

[0084] Embodiment 4:

[0085] As shown in Figure 5 The embodiment provides an electronic device, including a memory 51 and a processor 52, the memory 51 stores a computer program, and the processor 52 is configured to run the computer program to realize the processing method of the real-time label as described in embodiment 1 or embodiment 2.

[0086] Embodiment 5:

[0087] The embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the processing method of the real-time label as described in embodiment 1 or embodiment 2.

[0088] In summary, the processing method of the real-time label, the device, the electronic device and the computer readable storage medium provided by the embodiment of the present application are used to acquire the configuration relationship between the current user behavior and the data source when the current user behavior leading to the change of the user state is monitored, and extract the data of the real-time label of the user from the corresponding data source according to the configuration relationship to process and generate a new real-time label. Since the user label is monitored and generated in real time, the processing and extraction of the real-time label are not required when the real-time label is required by the business system, so that the real-time label has high real-time performance and ensures the accuracy of the real-time label. Secondly, the extracted data of the real-time label of the user is summarized as one piece of data, and then processed into a new real-time label to be transmitted to the business system, so that the network communication resources can be saved. Further, the new real-time label is written into the hbase in batches in the form of flink, so as to avoid a large amount of repeated calculation and ensure the stability of the system. Finally, the real-time label of the user is acquired in the database supporting high-concurrency query according to the query request of the business system, and the acquired real-time label of the user is fed back to the business system through an interface, which greatly improves the query efficiency of the real-time label, ensures the data storage safety, and can control the concurrency of the query to ensure the reliability of the system.

[0089] It can be understood that the above embodiments are only exemplary embodiments for illustrating the principles of the present application, and the present application is not limited thereto. Various modifications and improvements can be made by those skilled in the art without departing from the spirit and essence of the present application, and these modifications and improvements are also considered as the protection scope of the present application.

Claims

1. A method of processing real-time tags, characterized by, The method comprises the following steps: monitoring current user behavior that causes a change in user state; obtaining a configuration relationship between the current user behavior and a data source; extracting user real-time tag data from the corresponding data source according to the configuration relationship; processing the user real-time tag data to generate new real-time tags, storing the new real-time tags in a database supporting high-concurrency queries in real time, wherein the obtaining of the configuration relationship between the current user behavior and the data source comprises: obtaining the configuration relationship between the current user behavior and the data source from a preset database, wherein a distributed relational database stores user behavior, data sources, triggered user real-time tags, structured query language (SQL) used by the user real-time tags, and mapping relationships among the four, the number of triggered user real-time tags is one or more, the extracting of the user real-time tag data from the corresponding data source according to the configuration relationship comprises: executing an SQL statement to extract the user real-time tag data from the corresponding data source according to the configuration relationship.

2. The method of claim 1, wherein, The processing of the user real-time tag data to generate new real-time tags comprises: merging the user real-time tag data into one piece of data; performing data cleaning and processing on the merged data to generate new real-time tags.

3. The method of claim 2, wherein, After the storing of the new real-time tags in the database supporting high-concurrency queries in real time, the method further comprises: receiving a query request for user real-time tags; obtaining user real-time tags from the database supporting high-concurrency queries according to the query request.

4. A processing apparatus of a real-time tag, characterized by, The method comprises the following steps: a monitoring module for monitoring current user behavior that causes a change in user state, an obtaining module connected to the monitoring module for obtaining a configuration relationship between the current user behavior and a data source, an extracting module connected to the obtaining module for extracting user real-time tag data from the corresponding data source according to the configuration relationship, a processing module connected to the extracting module for processing the user real-time tag data to generate new real-time tags, a storage module connected to the processing module for storing the new real-time tags in a database supporting high-concurrency queries in real time, wherein the obtaining module is configured to obtain the configuration relationship between the current user behavior and the data source from a preset database, wherein a distributed relational database stores user behavior, data sources, triggered user real-time tags, structured query language (SQL) used by the user real-time tags, and mapping relationships among the four, the number of triggered user real-time tags is one or more, and the extracting module is configured to execute an SQL statement to extract the user real-time tag data from the corresponding data source according to the configuration relationship.

5. The apparatus of claim 4, wherein, The method further comprises a storage module, the storage module is connected to the processing module and is configured to store the new real-time tags in a database supporting high-concurrency queries in real time.

6. An electronic device, comprising: The method comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to implement the real-time tag processing method according to any one of claims 1-3.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements a real-time label processing method according to any one of claims 1-3.

Images