Data processing method and device based on real-time data, equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device based on real-time data, equipment and a storage medium. The method comprises the following steps: acquiring external behavior data; the external behavior data represents behaviors made by a user on an application system or data generated by a third-party system during operation; processing the behavior data according to preset operator information to obtain corresponding target data; the preset operator information represents operators for processing the behavior data and a processing sequence among the operators, and the processing sequence among the operators is obtained by dragging a worker on a preset interface; if the target data meets the preset user configuration condition, pushing the external behavior data to an application system, and sending prompt information; the user configuration condition represents a condition that the user is a potential consumption user, and the prompt information is used for prompting a worker to carry out consumption confirmation on the user or give an alarm to a third-party system. The method can reduce the development threshold, simplify the development process and improve the working efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for data processing based on real-time data. Background Art

[0002] With the development of the Internet and the Internet of Things, massive amounts of real-time data are generated. Staff need to analyze this data in real time to enhance their competitive advantage, such as analyzing customer behavior in real time to adjust recommendation strategies and analyzing transaction data in real time to optimize investment portfolios.

[0003] At present, the complexity of real-time data analysis is constantly increasing, but the traditional rule-based model calculation method has problems such as high development and use thresholds, large workload, and low data processing efficiency. It is no longer sufficient to meet the needs of staff for quick decision-making in business scenarios. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for data processing based on real-time data, so as to improve the efficiency of data processing.

[0005] In a first aspect, an embodiment of the present application provides a method for data processing based on real-time data, comprising:

[0006] According to a preset data acquisition operator, external behavior data is acquired; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation;

[0007] According to the preset data processing rule information, the external behavior data is processed to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operator for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by the staff dragging on the preset interface, and the target data represents the processed external behavior data;

[0008] If the target data meets the preset user configuration conditions, the target data is pushed to the application system according to the preset data output operator, and a prompt message is issued; wherein the preset user configuration conditions represent the conditions under which the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or to alert a third-party system.

[0009] In one possible implementation, the pre-set data processing rule information includes at least one of an input data connection operator, an output data connection operator, a dimension table data association operator, a data verification operator, a data compilation operator, a filtering operator, a deduplication operator, a splitting operator, a merging operator, a time series window aggregation operator, and a backtracking window aggregation operator.

[0010] In a possible implementation, the preset data processing rule information includes a data verification operator and a data compilation operator; according to the preset data processing rule information, the behavior data is processed to obtain target data corresponding to the behavior data, including:

[0011] According to the data verification operator, format verification is performed on the external behavior data to obtain a verification result;

[0012] If the verification result indicates that the external behavior data is in a preset data format, the target data is obtained according to the data compilation operator based on a preset parsing and running engine.

[0013] In a possible implementation manner, the preset data processing rule information further includes a filtering operator; and further includes:

[0014] If the verification result indicates that the external behavior data is not in a preset data format, determining data to be filtered from the external behavior data according to a preset filtering operator;

[0015] Filtering the data to be filtered to obtain filtered behavior data;

[0016] According to the filtered external behavior data and the data compilation operator, based on a preset parsing and running engine, the target data is obtained.

[0017] In a possible implementation manner, the preset data processing rule information also includes a deduplication operator; and further includes:

[0018] According to the deduplication operator, duplicate data is determined from the external behavior data, and the duplicate data is screened out.

[0019] In a possible implementation manner, the preset data processing rule information further includes a time series window aggregation operator and a backtracking window aggregation operator; and further includes:

[0020] According to the time series window aggregation operator and the backtracking window aggregation operator, the time length and position selected by the staff are determined, and the external behavior data is aggregated and calculated according to the time length and position; wherein the aggregation calculation includes at least one of sum calculation, count calculation, maximum value calculation, ranking calculation, sorting calculation, mean calculation, median calculation, filtering calculation in a specified order, and current limiting calculation.

[0021] In a possible implementation manner, the preset data processing rule information also includes a dimension table association operator, including:

[0022] Obtaining an identifier from the external behavior data; wherein the identifier represents a unique marking symbol of the data;

[0023] Searching a dimension data table associated with the identifier from a preset dimension table database as a target data table; wherein the preset dimension table database includes a plurality of dimension data tables, and the dimension data tables store attribute information of the user;

[0024] The attribute information of the user is obtained from the target data table, and the attribute information of the user is pushed to a target location; wherein the target location is a target database or a preset operator.

[0025] In a possible implementation, if the target data meets a preset user configuration condition, pushing the target data to the application system includes:

[0026] If the target data meets the preset user configuration conditions, determining the data type of the external behavior data;

[0027] According to the association relationship between the preset data types and the databases, determining the database corresponding to the data type of the external behavior data as the target database;

[0028] The target data is stored in the target database, and the target data is pushed to the application system.

[0029] In a possible implementation, it further includes:

[0030] Converting the preset user configuration information into initial condition information; wherein the initial condition information is in the form of a combination of JSON and a domain-specific language DSL; the preset user configuration information represents condition information that the user is a potential consumer user;

[0031] The initial condition information is verified and compiled to obtain the preset user configuration condition.

[0032] In a possible implementation, it further includes:

[0033] The processing result information and processing time information of each operator in the preset data processing rule information are stored in a state database; wherein the state database is used to store the result generated during the processing of each operator and the current time information.

[0034] In a possible implementation, it further includes:

[0035] In response to a staff member's instruction to update preset configuration information, determine updated configuration information; wherein the configuration information includes preset data processing rule information and preset user configuration conditions, and the configuration information is in the form of a combination of JSON and a domain-specific language DSL;

[0036] Processing the behavior data according to the updated data processing rule information to obtain target data corresponding to the behavior data;

[0037] If the target data meets the updated user configuration conditions, the target data is pushed to the application system and a prompt message is issued.

[0038] In a second aspect, an embodiment of the present application provides a device for data processing based on real-time data, including:

[0039] An acquisition module, which acquires external behavior data according to a preset data acquisition operator; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation;

[0040] A processing module processes the external behavior data according to preset data processing rule information to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operator for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by dragging on a preset interface by a staff member, and the target data represents the processed external behavior data;

[0041] The first push module pushes the target data to the application system according to the preset data output operator and issues a prompt message if the target data meets the preset user configuration conditions; wherein the preset user configuration conditions represent the conditions that the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or alert the third-party system.

[0042] In a third aspect, an embodiment of the present application provides a device for data processing based on an application program, comprising: a memory, a processor;

[0043] The memory stores computer-executable instructions;

[0044] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0047] The embodiment of the present application provides a method, device, equipment and storage medium for data processing based on real-time data. In this solution, external behavior data is obtained through a data interface, and various operators are preset. The staff sets the operator information by dragging the operator on the interface, wherein the preset operator information represents the operator for processing the behavior data and the processing order between the operators, so that data processing is performed according to the preset operator information. User configuration conditions are also preset. If the processed data meets the preset user configuration conditions, the external behavior data is pushed to the application system, and the staff confirms the consumption of potential consumers based on the information, saving manpower and time for data processing, improving the efficiency of data processing, and achieving the effect of lowering the development threshold, simplifying the development process, and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] Figure 1 A system overall architecture diagram of a data processing method based on real-time data provided by this application;

[0050] Figure 2 A system functional architecture diagram of a data processing method based on real-time data provided by this application;

[0051] Figure 3 A flowchart of a method for data processing based on real-time data provided in this application;

[0052] Figure 4 A flowchart of a method for data processing based on real-time data provided in this application;

[0053] Figure 5 A schematic diagram of the structure of a data processing device based on real-time data provided by the present application;

[0054] Figure 6 A schematic diagram of the structure of an electronic device for data processing based on real-time data provided in this application.

[0055] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0057] First, the terms involved in this application are explained:

[0058] Data stream: refers to the fact that in the natural environment, data is mostly generated in a streaming manner. Whether it is event data from a server, transaction data from a stock exchange, or data from various sensors, the data is generated continuously and never stops, forming a data stream.

[0059] Stream computing: It is a technology for processing data streams. Its core is to process data in real time and extract calculation results from data streams in real time to ensure that the value of data does not decrease with the passage of time.

[0060] Stateful computing: refers to the presence of state data in stream computing. For example, when calculating the total amount of all orders, the total amount of orders accumulated up to the current time is a state data.

[0061] Time window: refers to a time period, such as every 5 minutes, the last 10 minutes, the next hour, and this week. It is usually used with stateful computing to form statistical indicators.

[0062] Time series rules: refers to the stream computing rules generated by analyzing the data in the data stream according to the time series (or time window), also referred to as stream computing rules in this article.

[0063] Flink: Apache Flink is a distributed, high-performance big data stream computing framework that provides low-latency, high-throughput computing capabilities, good fault tolerance, and also supports stateful streaming computing.

[0064] With the development of the Internet and the Internet of Things, massive amounts of real-time data are generated. Staff need to analyze this data in real time to enhance their competitive advantage, such as analyzing customer behavior in real time to adjust recommendation strategies and analyzing transaction data in real time to optimize investment portfolios.

[0065] However, based on the description of the data processing method in the background technology, it can be seen that in the prior art, most staff are not familiar with the relevant technology, which makes it difficult to use it directly, the business response is slow, and the system needs to apply for a large amount of hardware resources for real-time computing operation and storage. In addition, the operation and maintenance of the system also requires a lot of manpower and energy.

[0066] At present, the complexity of real-time data analysis is also increasing. Traditional rule-based model computing does not support stateful computing and cannot perform real-time analysis of data flows within a time window. It is no longer sufficient to meet the staff's needs for quick decision-making in business scenarios. At the same time, how to lower the development and usage threshold, reduce workload and improve work efficiency is an issue that needs to be urgently addressed.

[0067] The present application provides a data processing method based on real-time data, which relates to the field of data processing, and in particular to a data processing method, device, equipment and storage medium based on real-time data, aiming to solve the above technical problems of the prior art.

[0068] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] Figure 1 The overall system architecture diagram of a data processing method based on real-time data provided by this application is as follows: Figure 1As shown in the figure, the submitted real-time tasks are deployed in a cloud-native mode based on the container cluster management system (Kubernetes, abbreviated as K8S), which realizes the distributed operation of real-time tasks and resource isolation between tasks. At the same time, the Flink data connector is further optimized to open up the data channel between real-time computing tasks and various big data clusters. Real-time data can be sent and received with various upstream and downstream systems through middleware such as distributed stream processing platform (Kafka) or streaming data lake storage technology (Paimon). In addition, real-time tasks use low-cost storage resources such as distributed file system (Hadoop Distributed File System, abbreviated as HDFS) and storage area (Simple Storage Service, abbreviated as S3) as state storage to meet the data storage requirements under large data volume and long time window. The dimension table data storage area uses an open source search engine (elasticsearch, abbreviated as ES) and a database management system (Oracle).

[0070] The staff or data engineer configures the input and output, data type, Chinese interpretation and other information of the data in the system in advance. After obtaining the user behavior data, the obtained user behavior data is input into the system through middleware such as Kafka and Paimon. The container cluster management system is deployed, and different real-time tasks can run simultaneously without affecting each other. The system is provided with a state storage area and a dimension table data storage area. The data generated within a certain period of time in the calculation are all state data, which are stored in the state storage area. The dimension table data storage area is used to store user attribute information. Finally, the information is output through middleware such as Kafka and ES. The information processed by the system can be displayed in the terminal system, which is convenient for the staff to view and confirm the consumption to the user.

[0071] Figure 2 A system functional architecture diagram of a data processing method based on real-time data provided in this application, such as Figure 2 As shown, the system is mainly divided into three parts, including data center, task development and instance operation and maintenance.

[0072] The data center is used to maintain the metadata of input and output data for upstream and downstream docking, such as the database to which the data belongs, the field name, Chinese meaning, data type and other information of the data. It is generally maintained by administrators or data engineers to facilitate users to understand the meaning of data when developing real-time tasks. The data center includes data source management module, output data management module, input data management module, dimension table data management module, grouping information management module, function management module, preset event library management module, etc.

[0073] Task development provides an easy-to-understand, process-orchestrated visual task development function. At the same time, it abstracts a variety of operators such as data input and output, data filtering, dimension table association, and timing rules related to time windows based on real-time common scenarios. The page can be operated without code throughout the process, which is convenient for business users to use directly. Task development includes visualization modules, expression parsing modules, process orchestration modules, version record modules, and operator modules, among which the operator modules include: data verification operators and data compilation operators, which are used to verify the data format and compile the data; input data connection operators and output data connection operators, which are used to connect to external databases, external data fields and other information; filtering operators, which are used to filter out useless data; deduplication operators, which are used to deduplicate duplicate data according to the data object identifiers configured by the staff, so as to eliminate duplicate data generated by external databases due to errors, retries, etc.; dimension table association operators, which are used to connect data in the dimension database with external The merge operator is used to merge data from different sources according to the merge conditions configured by the staff to execute the same subsequent processing logic; the split operator is used to split the data from the same source according to the split conditions configured by the staff, and send it to different subsequent data processing operators to execute different subsequent processing logic; the time series window aggregation operator and the backtracking window aggregation operator are used to sort, compare and aggregate the real-time streaming data of the specified time window segment in real time according to the operator internal processing logic such as the time window length, time window position, and data aggregation conditions configured by the staff, obtain the calculation results, and send them to the next data processing operator.

[0074] Instance operation and maintenance is mainly responsible for managing the deployment of tasks on the container cluster management system (Kubernetes, abbreviated as K8S) cloud, the allocation of task running resources, and the monitoring of running status. A task represents a complete computing process configured by the staff in the system for a certain business scenario, such as the computing process of the scenario of users placing orders online, the computing process of the scenario of users opening accounts, etc. Task rules represent preset operator information and preset user configuration conditions, where operator information includes operator computing rule information and arrangement order information. Instance operation and maintenance includes a task testing module for testing tasks, a deployment task module for deploying tasks, a task monitoring module for monitoring the execution process of tasks, a cancellation task module for canceling the running of tasks, a task resource management module for managing task-related resources, a task rule dynamic update module, which can update task rules in real time when tasks are running without restarting the tasks, and a task history status recovery module for viewing and restoring historical task status.

[0075] Figure 3 A flowchart of a method for data processing based on real-time data provided in this application is as follows: Figure 3As shown, the method includes:

[0076] S301. Acquire external behavior data according to a preset data acquisition operator; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation.

[0077] Exemplarily, the preset data acquisition operator is an input data connection operator, which is used to obtain external behavior data. An application for transactions may be installed in the terminal device, and the behavior data includes the operation behavior performed by the user on the application of the terminal device or the operation data of a third-party system, such as system operation logs and other data. Taking the user's online order as an example, the behavior data represents the amount, order number, product information, etc. of the user's online order. When the user performs transactions and other operations on the application, the system can obtain the user's behavior data in real time through the input data connection operator in the data connector, where the data connector is used to connect to external databases, external data fields and other information.

[0078] The beneficial effect of this setting is that the input data connection operator is used to obtain external behavior data, which facilitates flexible connection to external databases, real-time tracking of user operation status, and improved work efficiency.

[0079] S302. Process the external behavior data according to the preset data processing rule information to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operators for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by the staff dragging on the preset interface, and the target data represents the processed external behavior data.

[0080] For example, business scenarios are abstracted into operators, such as the business scenario of users placing orders online. By disassembling the calculation process, tasks are configured and generated in the form of operator combinations, where a task represents a complete calculation process configured by the staff in the system for a certain business scenario. The preset data processing rule information represents the operators that process external behavior data and the processing order between operators. The processing order between operators is obtained by the staff dragging on the preset interface. Figure 2 It can be seen that the staff can use the visualization module in task development to drag and drop the preset operators to edit the calculation rules for the scenario. For example, when the system obtains the user's behavior data in a certain scenario, it verifies, compiles, filters, removes duplicates, and performs time window aggregation analysis and calculation on the user's behavior data, and determines whether the data meets the user configuration conditions. If it meets the preset user configuration conditions, the processed data is determined to be the target data.

[0081] The beneficial effect of this setting is that it makes it easier for staff to understand the calculation rules and lowers the threshold for development and use.

[0082] In this embodiment, the pre-set data processing rule information includes at least one of an input data connection operator, an output data connection operator, a dimension table data association operator, a data verification operator, a data compilation operator, a filtering operator, a deduplication operator, a splitting operator, a merging operator, a time series window aggregation operator, and a backtracking window aggregation operator.

[0083] Exemplarily, the system contains a data connector and a data calculator, and the data calculator contains a variety of operators for computing and processing real-time data, including data filtering operators, data deduplication operators, stream data splitting operators, stream data merging operators, time series window aggregation calculation operators, and backtracking window aggregation calculation operators. The data connector is used to connect with external data, including input data connection operators, output data connection operators, dimension table data association operators, etc. Among them, the input data connection operator and the output data connection operator are used for the docking external database, external data field and other information; the data verification operator and the data compilation operator are used to verify the data format and compile the data; the filtering operator is used to filter out useless data; the deduplication operator is used to deduplicate the duplicate data according to the data object identifier configured by the staff to exclude the duplicate data generated by the external docking database due to errors, retries and the like; the dimension table data association operator is used to associate the data in the dimension database with the external behavior data; the merging operator is used to merge the data from different sources according to the merging conditions configured by the staff to execute the same subsequent processing logic; the splitting operator is used to split the data from the same source according to the splitting conditions configured by the staff and send it to different subsequent data processing operators to execute different subsequent processing logic; the time series window aggregation operator and the backtracking window aggregation operator are used to perform real-time sorting, comparison and aggregation calculation on the real-time stream data of the specified time window segment according to the operator internal processing logic such as the time window length, time window position, and data aggregation conditions configured by the staff, obtain the calculation result, and send it to the next data processing operator.

[0084] The beneficial effect of such a setting is that the system is provided with a variety of operators with rich functions, which makes it convenient for the staff to use operators with different functions to arbitrarily combine them to complete a variety of complex calculation processes and improve work efficiency.

[0085] In this embodiment, the preset data processing rule information includes a data verification operator and a data compilation operator; according to the preset data processing rule information, the behavior data is processed to obtain target data corresponding to the behavior data, including: according to the data verification operator, the external behavior data is format-checked to obtain a verification result; if the verification result indicates that the external behavior data is in a preset data format, according to the data compilation operator, based on a preset parsing and running engine, the target data is obtained.

[0086] Exemplarily, the preset data processing rule information represents the operators for processing the behavior data and the processing order between the operators. A variety of operators are preset, such as filtering operators, deduplication operators, splitting operators, merging operators, time series window operators, backtracking window operators, data verification operators, data compilation operators, etc., wherein the data verification operator contains preset verification rules, and the data compilation operator contains the code for compiling the data, which is used to compile the acquired behavior data. In task development, the acquired user behavior data is processed by dragging operators. The system can support the processing of multiple computing tasks, and each computing task can be different and independent of each other. For example, the dragged operators include the check operator and the compile operator. The compile operator can be dragged after the check operator. In the process of data processing, the behavior data can be pre-checked by the check operator to ensure that the business can proceed smoothly. The check operator can contain preset check rules, and the data format is checked according to the preset check rules. For example, the rule uses a non-existent data field to cause a program exception, and compares the string "09" with the number "1" to cause an error in the result. After the check is completed, the compile operator can be used to compile the checked data.

[0087] The beneficial effect of such a setting is to ensure that the business in the system can run smoothly and efficiently, reduce the possibility of task failure and business logic errors due to data problems, and improve the efficiency of data processing.

[0088] In this embodiment, the preset data processing rule information also includes a filtering operator; and also includes: if the verification result indicates that the external behavior data is not in a preset data format, then according to the preset filtering operator, the data to be filtered is determined from the external behavior data; the data to be filtered is filtered to obtain filtered behavior data; according to the filtered external behavior data and the data compilation operator, the target data is obtained based on a preset parsing and running engine.

[0089] Exemplarily, the filtering operator is a data processing method preset in the system, which is used to filter out useless data. The system is also pre-set with a verification operator and a compilation operator, which are used to verify the data format and compile the data. For example, the acquired user behavior data is processed, and the behavior data is first verified by the verification operator. If the verification result does not meet the preset verification rules, that is, the verification result does not meet the preset data format, then the data to be filtered to be filtered can be determined from the behavior data according to the preset filtering operator, and the data to be filtered is filtered to obtain the filtered behavior data. Then, according to the preset compilation operator and the parsing and running engine, the filtered behavior data is compiled and run to ensure that the business can be executed efficiently and finally the target data is obtained.

[0090] The beneficial effect of such a setting is that it is easy to filter out useless data, reduce workload and improve work efficiency.

[0091] In this embodiment, the preset data processing rule information also includes a deduplication operator; and further includes: determining duplicate data from the external behavior data according to the deduplication operator, and filtering out the duplicate data.

[0092] Exemplarily, the preset data processing rule information represents the operators for processing external behavior data and the processing order between the operators. The processing order between the operators is obtained by dragging on the preset interface by the staff. The deduplication operator is used to deduplicate duplicate data to eliminate duplicate data generated by errors, retries, etc. in the external docking database. According to the deduplication operator in the system, the duplicate data in the acquired external behavior data is determined, and the deduplication operator is used to screen out the duplicate data to remove the duplicate data.

[0093] The beneficial effect of such a setting is that it is easy to remove duplicate data and eliminate duplicate data generated by errors, retries, etc. in the external docking database when the system is running, thereby improving work efficiency.

[0094] In this embodiment, the preset data processing rule information also includes a time series window aggregation operator and a backtracking window aggregation operator; it also includes: according to the time series window aggregation operator and the backtracking window aggregation operator, determining the time length and position selected by the staff, and performing aggregation calculation on the external behavior data according to the time length and position; wherein the aggregation calculation includes at least one of sum calculation, count calculation, maximum value calculation, ranking calculation, sorting calculation, mean calculation, median calculation, filtering calculation in a specified order, and current limiting calculation.

[0095] Exemplarily, the preset data processing rule information represents the operators that process the external behavior data and the processing order between the operators. The processing order between the operators is obtained by dragging on the preset interface by the staff. The time series window aggregation operator and the backtracking window aggregation operator are used to perform real-time sorting, comparison and aggregation calculations on the real-time streaming data of the specified time window segment in accordance with the operator internal processing logic such as the time window length, time window position, and data aggregation conditions configured by the staff, obtain the calculation results, and send them to the next data processing operator. For example, after obtaining the external behavior data, the behavior data is aggregated according to the time series window aggregation operator and the backtracking window aggregation operator, where the aggregation calculation includes multiple types, such as sum calculation, count calculation, maximum value calculation, ranking calculation, sorting calculation, mean calculation, median calculation, filtering calculation in specified order, current limiting calculation, etc.

[0096] The beneficial effect of this setting is that setting the time series window aggregation operator and the backtracking window aggregation operator facilitates real-time sorting, comparison and aggregation calculation of real-time streaming data in the specified time window segment, reduces the amount of calculation, and accurately obtains data within the specified time period.

[0097] In this embodiment, the preset data processing rule information also includes a dimension table association operator, including: obtaining an identifier from external behavior data; wherein the identifier represents a unique marking symbol of the data; searching a dimension data table associated with the identifier from a preset dimension table database, which is a target data table; wherein the preset dimension table database includes multiple dimension data tables, and the dimension data tables store user attribute information; obtaining the user attribute information from the target data table, and pushing the user attribute information to a target location; wherein the target location is a target database or a preset operator.

[0098] Exemplarily, the preset data processing rule information represents the operators that process the external behavior data and the processing order between the operators. The processing order between the operators is obtained by the staff dragging on the preset interface. The dimension table association operator is used to associate the acquired external behavior data with the data in the dimension table database. For example, a dimension table database is preset in the system, and the identifier is used to mark the external data. The identifier corresponds to the code in the dimension table database. The dimension table database contains multiple dimension data tables. The dimension data table is used to store the user's attribute information. The dimension table database and the dimension data table both contain the code corresponding to the identifier, and the code corresponds to the identifier one by one. Finally, the system sends a prompt message or issues an alarm to the third-party system. After the staff obtains the prompt message, they confirm the user's consumption through the attribute information in the dimension data table.

[0099] The beneficial effect of this setting is that the behavioral data is stored and associated with the user's personal information, making it easier for staff to quickly find user information and improve work efficiency.

[0100] S303. If the target data meets the preset user configuration conditions, the target data is pushed to the application system according to the preset data output operator, and a prompt message is issued; wherein the preset user configuration conditions represent the conditions under which the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or to alert the third-party system.

[0101] Exemplarily, the user configuration condition represents the condition under which the user is a potential consumer user. The user configuration condition is pre-configured in the system by the staff according to different business scenarios. Potential consumer users are users who are very likely to consume. The prompt information is used to prompt the staff to confirm the user's consumption or alert the third-party system. For example, taking the user's online order as an example, when the user configuration condition is pre-set to a certain amount threshold, when the user's order amount meets the pre-set amount threshold, it is considered that the target data meets the user configuration condition, and the system stores the target data and the status information during the data processing process, wherein the status information refers to the data information generated at any time during the data processing process, and finally sends a prompt message to the staff. After the staff obtains the information, they confirm the user's consumption, such as by outbound calls, or risk control, etc. In addition, an alarm message can also be generated in the third-party system for prompts.

[0102] The beneficial effect of such a setting is that setting user configuration conditions facilitates real-time tracking of the behavior status of potential consumer users, obtaining user behavior data information, reducing workload and improving work efficiency.

[0103] In this embodiment, external behavior data is obtained through the data interface, and various operators are preset. The staff sets the operator information by dragging the operator on the interface, wherein the preset operator information represents the operator that processes the behavior data and the processing order between the operators, so that data processing is performed according to the preset operator information. User configuration conditions are also preset. If the processed data meets the preset user configuration conditions, the behavior data will be pushed to the staff, and the staff will confirm the consumption of potential consumer users based on the information, so as to save manpower and time for data processing, improve the efficiency of data processing, and achieve the effect of lowering the development threshold, simplifying the development process, and improving work efficiency.

[0104] Figure 4 A flowchart of a method for data processing based on real-time data provided in this application is shown in FIG. Figure 4 As shown, in this embodiment Figure 3 Based on the embodiment, a method for data processing based on real-time data is described in detail, and the method includes:

[0105] S401. Acquire external behavior data according to a preset data acquisition operator; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation.

[0106] Exemplarily, this step refers to the above-mentioned S301 and will not be described in detail.

[0107] S402. Process the external behavior data according to the preset data processing rule information to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operators for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by the staff dragging on the preset interface, and the target data represents the processed external behavior data.

[0108] Exemplarily, this step refers to the above-mentioned S302 and will not be described in detail.

[0109] S403: If the target data meets the preset user configuration conditions, determine the data type of the external behavior data.

[0110] Exemplarily, the user configuration condition represents the condition that the user is a potential consumer user. The user configuration condition is pre-configured in the system by the staff according to different business scenarios. Potential consumer users are users who are likely to consume. The prompt information is used to prompt the staff to confirm the user's consumption or alert the third-party system. The data type is divided into state data and result data. The state data is stored in the state database, and the result data is stored in the target database. If the target data meets the user configuration condition, the data type of the data is determined. For example, taking the user's online order as an example, the user's order information is real-time data, then the data generated by the information during the calculation process is state data, and the target data formed after being processed by each operator is the result data.

[0111] The beneficial effect of this arrangement is that it facilitates the classification and storage of data and improves the working efficiency of the system.

[0112] S404: According to the association relationship between the preset data type and the database, determine the database corresponding to the data type of the external behavior data as the target database.

[0113] Exemplarily, the types of databases are divided into state databases and target databases, and different data are stored in corresponding databases according to the type of database. The acquired external behavior data and the database both contain codes, and the codes in the behavior data correspond to the codes in the database. When the external behavior data is acquired, the corresponding database can be found according to the codes in the behavior data. Among them, the results generated during the processing of each operator and the current time information are stored in the state database, and the target data obtained after processing is stored in the target database.

[0114] The beneficial effect of such a setting is that it facilitates real-time storage of a large amount of user status data and improves the working efficiency of the system.

[0115] S405: Store the target data in the target database, and push the target data to the application system.

[0116] Exemplarily, after obtaining external behavior data, the corresponding target database can be found according to the code in the behavior data. For example, taking a user order as an example, after obtaining the user's behavior data, the data is processed according to the preset operators, including verification, compilation, filtering, deduplication, time window aggregation calculation and other processing, to obtain the target data, find the corresponding target database according to the code, store the target data in the target database, and push the target data to the application system, including sending prompt information or alarm information to the staff or third-party system. In addition, the target data can also be associated with the data in the dimension table database through the dimension table association operator, and the staff can obtain the user's behavior data information and personal information. According to the user's personal information, the staff can confirm the consumption of the user through outbound calls or risk control.

[0117] The beneficial effect of such a setting is that it facilitates the storage of processed user behavior data and improves the working efficiency of the system.

[0118] In this embodiment, external behavior data is obtained through the data interface, and various operators are preset. The staff sets the operator information by dragging the operator on the interface, wherein the preset operator information represents the operator that processes the behavior data and the processing order between the operators, so that data processing is performed according to the preset operator information. User configuration conditions are also preset. If the processed data meets the preset user configuration conditions, the behavior data will be pushed to the staff, and the staff will confirm the consumption of potential consumer users based on the information, so as to save manpower and time for data processing, improve the efficiency of data processing, and achieve the effect of lowering the development threshold, simplifying the development process, and improving work efficiency.

[0119] In this embodiment, the preset user configuration information is converted into initial condition information; wherein the initial condition information is in the form of a combination of JSON and a domain-specific language DSL; the preset user configuration information represents condition information that the user is a potential consumer user; the initial condition information is verified and compiled to obtain the preset user configuration conditions.

[0120] Exemplarily, the initial condition information is in the form of a combination of JSON and a domain-specific language DSL. The preset user configuration information represents the condition information that the user is a potential consumer user. JSON plus expressions can realize the separation of real-time task rules and program codes, which is clearer to write and read, and is also convenient for integration with other systems. The task rules refer to the configuration of operator information, the arrangement order of operator sequence, and the setting of user configuration conditions by staff in a certain business scenario. The combination of JSON and a domain-specific language DSL is used to parse, verify, compile and execute data processing logic for user data. The DSL syntax expression provides a wealth of pre-verification rules and a complete anti-vulnerability injection system at the parsing layer, and realizes efficient execution of rules through pre-compilation at the running layer. At the same time, the DSL syntax expression carries information such as window mode and window duration, and supports dynamic update of time windows, where a time window refers to a time period, such as every 5 minutes, the last 10 minutes, the next 1 hour, until this week, etc. After obtaining the external behavior data, before processing the data, the user configuration information is converted into initial condition information, and the JSON and domain-specific language DSL syntax expressions are checked for data legitimacy, data format, processing logic, etc. to obtain the verification results. If the verification passes, the preset user configuration conditions are obtained.

[0121] The beneficial effect of such a setting is that it facilitates the pre-verification and compilation of DSL expressions, avoids system operation errors due to format errors, and improves work efficiency.

[0122] In this embodiment, the processing result information and processing time information of each operator in the preset data processing rule information are stored in the state database; wherein the state database is used to store the result generated during the processing of each operator and the current time information.

[0123] Exemplarily, the preset data processing rule information represents the operator for processing the behavior data and the processing order between the operators. The system is preset with a database for storing state data, which is used to store the results generated during the processing of each operator and the current time information. The database storing the state data is determined as the state database. The system includes a variety of operators for data calculation, such as verification operators, compilation operators, filtering operators, deduplication operators, splitting operators, merging operators, time series window aggregation calculation operators, backtracking window aggregation calculation operators, etc. In the process of processing the user's behavior data, including the verification, compilation, filtering, deduplication, splitting, merging, aggregation calculation and other processing processes of external behavior data information, each operator's processing results and time information of the data will be stored in the state database in real time.

[0124] The beneficial effect of such a setting is that the processing results and time information are stored in the state database in real time, which facilitates the storage of a large amount of processing data.

[0125] In this embodiment, in response to a staff member's update instruction for preset configuration information, updated configuration information is determined; wherein the configuration information includes preset data processing rule information and preset user configuration conditions, and the configuration information is in the form of a combination of JSON and a domain-specific language DSL language; according to the updated data processing rule information, the behavior data is processed to obtain target data corresponding to the behavior data; if the target data meets the updated user configuration conditions, the target data is pushed to the application system and a prompt message is issued.

[0126] Exemplarily, the configuration information includes preset operator information and preset user configuration conditions, wherein the operator information includes the calculation rule information and arrangement order information of the operator, and the user configuration condition represents the condition that the user is a potential consumer user, and the user configuration condition is pre-configured in the system by the staff according to different business scenarios, and the potential consumer user is a user who is likely to consume. The prompt information is used to prompt the staff to confirm the user's consumption or to warn the third-party system. For example, when the staff updates the configuration information, the system can respond to the staff's update instruction in real time during the task operation to determine the updated configuration information, wherein the configuration information is in the form of a combination of JSON and a domain-specific language DSL language, and the updated configuration information includes the update of the operator information and its arrangement order, the update of the user configuration condition, etc. According to the updated operator information, the user's behavior data is processed, including verification, compilation, filtering, deduplication, splitting, merging, time series window aggregation, etc., to obtain the target data corresponding to the behavior data. If the target data meets the updated user configuration conditions, the system stores the data in the target database and sends a prompt message. The staff confirms the consumption to the user according to the background data information.

[0127] The beneficial effect of such a setting is that it facilitates the system to quickly update the configuration information, improves timeliness, and reduces the possibility of task abnormalities caused by restarting the task.

[0128] Figure 5 A schematic diagram of a device structure for data processing based on real-time data provided in this application, such as Figure 5 As shown, a data processing device 500 based on real-time data provided in this embodiment includes:

[0129] The acquisition module 501 is used to acquire external behavior data according to a preset data acquisition operator; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation;

[0130] The processing module 502 is used to process the external behavior data according to the preset data processing rule information to obtain the target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operator for processing the external behavior data and the processing order between the operators, and the processing order between the operators is obtained by the staff dragging on the preset interface, and the target data represents the processed external behavior data;

[0131] The first push module 503 is used to push the target data to the application system and issue a prompt message according to a preset data output operator if the target data meets the preset user configuration conditions; wherein the preset user configuration conditions represent the conditions under which the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or alert the third-party system.

[0132] In one example, the processing module 502 includes: pre-set data processing rule information including at least one of an input data connection operator, an output data connection operator, a dimension table data association operator, a data verification operator, a data compilation operator, a filtering operator, a deduplication operator, a splitting operator, a merging operator, a time series window aggregation operator, and a backtracking window aggregation operator.

[0133] In one example, the processing module 502 includes:

[0134] A verification unit, configured to perform format verification on the external behavior data according to the data verification operator to obtain a verification result;

[0135] A compiling unit is used to obtain the target data according to the data compiling operator and based on a preset parsing and running engine if the verification result indicates that the external behavior data is in a preset data format.

[0136] In one example, the preset data processing rule information also includes a filtering operator; the processing module 502 includes:

[0137] A first determining unit, configured to determine data to be filtered from the behavior data according to a preset filtering operator if the verification result indicates that the behavior data is not in a preset data format;

[0138] A filtering unit, used for filtering the data to be filtered to obtain filtered behavior data;

[0139] The second determining unit is used to obtain the target data according to the filtered behavior data and the compilation operator based on a preset parsing engine.

[0140] In one example, the preset data processing rule information also includes a deduplication operator, and the processing module 502 includes:

[0141] A deduplication unit is used to determine duplicate data from the external behavior data according to the deduplication operator, and to filter out the duplicate data.

[0142] In one example, the preset data processing rule information also includes a time series window aggregation operator and a backtracking window aggregation operator, and the processing module 502 further includes:

[0143] A window calculation unit is used to determine the time length and position selected by the staff according to the time series window aggregation operator and the backtracking window aggregation operator, and perform aggregation calculation on the external behavior data according to the time length and position; wherein the aggregation calculation includes at least one of sum calculation, count calculation, maximum value calculation, ranking calculation, sorting calculation, mean calculation, median calculation, filtering calculation in a specified order, and current limiting calculation.

[0144] In one example, the preset data processing rule information also includes a dimension table association operator, and the processing module 502 includes:

[0145] An acquisition unit, used to acquire an identifier from the external behavior data; wherein the identifier represents a unique marking symbol of the data;

[0146] A search unit, used to search a dimension data table associated with the identifier from a preset dimension table database, which is a target data table; wherein the preset dimension table database includes a plurality of dimension data tables, and the dimension data tables store attribute information of the user;

[0147] The first pushing unit is used to obtain the attribute information of the user from the target data table and push the attribute information of the user to a target location; wherein the target location is a target database or a preset operator.

[0148] In one example, the first push module 503 includes:

[0149] A third determining unit, configured to determine a data type of the external behavior data if the target data meets a preset user configuration condition;

[0150] a fourth determining unit, configured to determine, according to a preset association relationship between the data type and the database, a database corresponding to the data type of the external behavior data as a target database;

[0151] The second pushing unit is used to store the target data in the target database and push the target data to the application system.

[0152] In one example, it also includes:

[0153] The conversion module is used to convert the preset user configuration information into initial condition information; wherein the initial condition information is in the form of a combination of JSON and a domain-specific language DSL; the preset user configuration information represents the condition information that the user is a potential consumer user.

[0154] In one example, it also includes:

[0155] The storage module stores the processing result information and processing time information of each operator in the preset data processing rule information into a state database; wherein the state database is used to store the result generated during the processing of each operator and the current time information.

[0156] In one example, it also includes:

[0157] A first confirmation module is used to determine updated configuration information in response to a staff member's update instruction for preset configuration information; wherein the configuration information includes preset data processing rule information and preset user configuration conditions, and the configuration information is in the form of a combination of JSON and a domain-specific language DSL;

[0158] A second confirmation module is used to process the behavior data according to the updated data processing rule information to obtain target data corresponding to the behavior data;

[0159] The second push module is used to push the target data to the application system and issue a prompt message if the target data meets the updated user configuration conditions.

[0160] This embodiment provides a data processing device based on real-time data, which can execute the method provided by the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0161] Figure 6 This is a schematic diagram of an electronic device structure based on real-time data processing provided by this application. Figure 5 As shown, the electronic device 600 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 600 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0162] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0163] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0164] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0165] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0166] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0167] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0168] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0169] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0170] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0171] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0174] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0175] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0176] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for data processing based on real-time data, characterized in that: include: According to a preset data acquisition operator, external behavior data is acquired; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation; According to the preset data processing rule information, the external behavior data is processed to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operator for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by the staff dragging on the preset interface, and the target data represents the processed external behavior data; If the target data meets the preset user configuration conditions, the target data is pushed to the application system according to the preset data output operator, and a prompt message is issued; wherein the preset user configuration conditions represent the conditions under which the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or to alert a third-party system.

2. The method according to claim 1, characterized in that The pre-set data processing rule information includes at least one of an input data connection operator, an output data connection operator, a dimension table data association operator, a data verification operator, a data compilation operator, a filtering operator, a deduplication operator, a splitting operator, a merging operator, a time series window aggregation operator, and a backtracking window aggregation operator.

3. The method according to claim 2, characterized in that The preset data processing rule information includes a data verification operator and a data compilation operator; According to the preset data processing rule information, the behavior data is processed to obtain target data corresponding to the behavior data, including: According to the data verification operator, format verification is performed on the external behavior data to obtain a verification result; If the verification result indicates that the external behavior data is in a preset data format, the target data is obtained according to the data compilation operator based on a preset parsing and running engine.

4. The method according to claim 3, characterized in that The preset data processing rule information also includes a filtering operator; and also includes: If the verification result indicates that the external behavior data is not in a preset data format, determining data to be filtered from the external behavior data according to a preset filtering operator; Filtering the data to be filtered to obtain filtered behavior data; According to the filtered external behavior data and the data compilation operator, based on a preset parsing and running engine, the target data is obtained.

5. The method according to claim 2, characterized in that: The preset data processing rule information also includes a deduplication operator; and also includes: According to the deduplication operator, duplicate data is determined from the external behavior data, and the duplicate data is screened out.

6. The method according to claim 2, characterized in that The preset data processing rule information also includes a time series window aggregation operator and a backtracking window aggregation operator; and also includes: According to the time series window aggregation operator and the backtracking window aggregation operator, the time length and position selected by the staff are determined, and the external behavior data is aggregated and calculated according to the time length and position; wherein the aggregation calculation includes at least one of sum calculation, count calculation, maximum value calculation, ranking calculation, sorting calculation, mean calculation, median calculation, filtering calculation in a specified order, and current limiting calculation.

7. The method according to claim 2, characterized in that: The preset data processing rule information also includes dimension table association operators, including: Obtaining an identifier from the external behavior data; wherein the identifier represents a unique marking symbol of the data; Searching a dimension data table associated with the identifier from a preset dimension table database as a target data table; wherein the preset dimension table database includes a plurality of dimension data tables, and the dimension data tables store attribute information of the user; The attribute information of the user is obtained from the target data table, and the attribute information of the user is pushed to a target location; wherein the target location is a target database or a preset operator.

8. The method according to claim 1, characterized in that: If the target data meets the preset user configuration conditions, the target data is pushed to the application system, including: If the target data meets the preset user configuration conditions, determining the data type of the external behavior data; According to the association relationship between the preset data types and the databases, determining the database corresponding to the data type of the external behavior data as the target database; The target data is stored in the target database, and the target data is pushed to the application system.

9. The method according to claim 1, characterized in that: The method further comprises: Converting the preset user configuration information into initial condition information; wherein the initial condition information is in the form of a combination of JSON and a domain-specific language DSL; the preset user configuration information represents condition information that the user is a potential consumer user; The initial condition information is verified and compiled to obtain the preset user configuration condition.

10. The method according to claim 1, characterized in that Also includes: The processing result information and processing time information of each operator in the preset data processing rule information are stored in a state database; wherein the state database is used to store the result generated during the processing of each operator and the current time information.

11. The method according to any one of claims 1 to 10, characterized in that Also includes: In response to a staff member's instruction to update preset configuration information, determine updated configuration information; wherein the configuration information includes preset data processing rule information and preset user configuration conditions, and the configuration information is in the form of a combination of JSON and a domain-specific language DSL; Processing the behavior data according to the updated data processing rule information to obtain target data corresponding to the behavior data; If the target data meets the updated user configuration conditions, the target data is pushed to the application system and a prompt message is issued.

12. A data processing device based on real-time data, characterized in that: include: An acquisition module, which acquires external behavior data according to a preset data acquisition operator; wherein the external behavior data represents the behavior performed by the user on the application system or the data generated by the third-party system during operation; A processing module processes the external behavior data according to preset data processing rule information to obtain target data corresponding to the external behavior data; wherein the preset data processing rule information represents the operator for processing the external behavior data and the processing order between the operators, the processing order between the operators is obtained by dragging on a preset interface by a staff member, and the target data represents the processed external behavior data; The first push module pushes the target data to the application system according to the preset data output operator and issues a prompt message if the target data meets the preset user configuration conditions; wherein the preset user configuration conditions represent the conditions that the user is a potential consumer user, and the prompt message is used to prompt the staff to confirm the user's consumption or alert the third-party system.

13. A device for data processing based on real-time data, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 11 when being executed by a processor.

Images