Business data processing method and device, display method and device, electronic equipment and medium

Abstract

Embodiments of the present application provide a service data processing method, a service data display method, an apparatus, an electronic device and a storage medium. The processing method comprises: acquiring service data in real time; storing the service data into a corresponding message queue; performing data processing on the service data in each message queue according to a time dimension to obtain a data characteristic value within a corresponding time; and sending the data characteristic value within the corresponding time to a first intermediate storage container for storage, so that when the first intermediate storage container receives a request for accessing a data value from an online service system or a third-party service system, the first intermediate storage container calculates a data value within a corresponding time window according to the data characteristic value and feeds back the data value to the online service system or the third-party service system. Thus, efficient access support is provided for the online service system or the third-party service system, and the real-time calculation accuracy of the online service system or the third-party service system is improved.

Description

Technical Field

[0001] This invention relates to the field of network technology, and in particular to a business data processing method, a business data display method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] In existing technologies, open-source real-time computing frameworks such as Spark Streaming, Flink, and Storm are commonly used for real-time metric calculation. The general idea behind these frameworks is to monitor message queue data in real time and calculate data within a unit time window using a fixed sliding step size. In particular, it's necessary to intercept users who use web crawlers to obtain large amounts of information from websites. For current website business scenarios, it's typically necessary to collect user behavior data in real time and calculate the number of pages viewed by users. When it's detected that the number of pages viewed by the same user within a certain time period exceeds a certain threshold (e.g., 500 pages), it's necessary to promptly intercept them.

[0003] The following example uses real-time computation with Spark Streaming. Assuming a time window of 1 hour, a maximum threshold of 500 pages viewed by a user, and a sliding step of 15 minutes, Spark Streaming will calculate the value for the most recent hour every 15 minutes. If a user has viewed 499 pages at 10:00, this time point will show no anomalies. When the time point reaches 10:10, the number of pages viewed reaches 600. The system will only store the value from 10:00, and 10:15 will be the next calculation point. In other words, although the user has reached the maximum threshold for page views, it hasn't reached the next calculation point in the time window. Therefore, the system will not process this request and will not intercept the user.

[0004] Therefore, in related technologies, the mismatch between calculation time and event time leads to errors in the calculation of the number of web pages viewed by users, thereby reducing the accuracy of real-time data calculation. Summary of the Invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a business data processing method and a business data display method to solve the technical problem in the prior art that the data calculation volume is inaccurate due to the mismatch between calculation time and event time and the error in data calculation volume.

[0006] Accordingly, embodiments of the present invention also provide a business data processing device and a business data display device to ensure the implementation and application of the above methods.

[0007] To solve the above problems, the present invention is achieved through the following technical solution:

[0008] The first aspect provides a business data processing method, including:

[0009] Real-time acquisition of business data;

[0010] The business data is stored in the corresponding message queue;

[0011] The business data in each message queue is processed according to the time dimension to obtain the data feature values ​​for the corresponding time period.

[0012] The data feature values ​​within the corresponding time period are sent to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container calculates the data value within the corresponding time window based on the data feature values ​​and feeds back the data value to the online business system or the third-party business system.

[0013] Optionally, after processing the business data in each message queue according to the time dimension to obtain the data feature values ​​for the corresponding time period, the method further includes:

[0014] The data feature values ​​within the corresponding time period are accumulated and / or deduplicated.

[0015] The results of the cumulative calculation and / or deduplication calculation are sent to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data values ​​within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed back the data values ​​to the online business system or the third-party business system.

[0016] Optionally, the step of accumulating and / or deduplicating the data feature values ​​within the corresponding time period includes:

[0017] The data feature values ​​within the corresponding time period are sent to a first delayed queue composed of a Kafka architecture. The first delayed queue stores the data feature values ​​according to their corresponding timestamps and then notifies the first computing center to perform cumulative calculations on the data feature values. Additionally, the first delayed queue manages the expiration of the received data feature values ​​and sends expired data feature values ​​to the first computing center. The first computing center then subtracts the expired data feature values ​​from the cumulative calculation result; and / or

[0018] The data feature values ​​within the corresponding time period are sent to a sequential queue composed of an intermediate storage container set and a second delayed queue composed of Kafka containers, respectively. The sequential queue and the second delayed queue store the data feature values ​​according to their corresponding timestamps. The sequential queue prunes the stored data feature values ​​according to the window size and notifies the second computing center after pruning to calculate the number of remaining data feature values ​​in the current sequential queue. When the sequential queue receives an expired message from the second delayed queue, it prunes the data according to the window size based on the expired message and notifies the second computing center after pruning to calculate the number of remaining data feature values ​​in the current queue.

[0019] The second aspect provides a method for displaying business data, including:

[0020] Received a query request from a user regarding their online business operations;

[0021] Based on the query request, the intermediate storage container requests the data value of the current user's online business operation; so that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation within the current time window based on the data feature value stored for each time period, and then feed back the data value.

[0022] Receive the data value of the user's online service operation within the current time window from the intermediate storage container;

[0023] Display the data values ​​of the user's online business operations so as to determine whether to process the user's online business operations based on the data values.

[0024] The third aspect provides a business data processing apparatus, including:

[0025] The acquisition module is used to acquire business data in real time.

[0026] The first sending module is used to send the business data to the corresponding message queue;

[0027] The processing module is used to process the business data in each message queue according to the time dimension to obtain the data feature values ​​within the corresponding time period;

[0028] The second sending module is used to send the data feature value obtained by the processing module within a corresponding time period to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container calculates the data value within the corresponding time window based on the data feature value and feeds back the data value to the online business system or the third-party business system.

[0029] Optionally, the device further includes:

[0030] The calculation module is used to perform cumulative calculation and / or deduplication calculation on the data feature values ​​within the corresponding time period after the processing module obtains the data feature values ​​within the corresponding time period.

[0031] The third sending module is used to send the results of the cumulative calculation and / or deduplication calculation calculated by the calculation module to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data values ​​within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed back the data values ​​to the online business system or a third-party business system.

[0032] Optionally, the calculation module includes: a fourth sending module and an accumulation calculation module; and / or, a fifth sending module and a deduplication calculation module, wherein,

[0033] The fourth sending module is used to send the data feature values ​​within the corresponding time period to the first delay queue composed of Kafka, and the first delay queue stores the data feature values ​​according to the corresponding timestamps and then notifies the first computing center.

[0034] The cumulative calculation module is used to perform cumulative calculation on the data feature value through the first computing center, and to perform expiration management on the received data feature value through the first delay queue, and send the expired data feature value to the first computing center, so that the first computing center subtracts the expired data feature value from the result of the cumulative calculation.

[0035] The fifth sending module is used to send the data feature values ​​within the corresponding time period to a sequential queue composed of an intermediate storage container set and a second delayed queue composed of Kafka containers, respectively; the sequential queue and the second delayed queue store the data feature values ​​according to the corresponding timestamps, respectively.

[0036] The deduplication calculation module is used to prune the stored data feature values ​​according to the window size through the sequential queue, and after the pruning is completed, notify the second computing center to calculate the number of remaining data feature values ​​in the current sequential queue; and when the sequential queue receives an expired message sent by the second delayed queue, it prunes the data according to the window size based on the expired message, and after the pruning is completed, notifies the second computing center to calculate the number of remaining data feature values ​​in the current queue.

[0037] The fourth aspect provides a business data display device, including:

[0038] The first receiving module is used to receive query requests from users regarding their online business operations.

[0039] The request module is used to request the data value of the current user's online business operation from the intermediate storage container according to the query request; so that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation in the current time window according to the data feature value stored in each time period, and feed back the data value.

[0040] The second receiving module is used to receive the data value of the user's online business operation within the current time window from the intermediate storage container;

[0041] The display module is used to display the data values ​​of the user's online business operations, so as to determine whether to process the user's online business operations based on the data values.

[0042] The fifth aspect provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the business data processing method as described above or the business data display method as described above.

[0043] The sixth aspect provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the business data processing method described above or the business data display method described above.

[0044] A seventh aspect provides a computer program product, wherein when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device performs the business data processing method described above or the business data display method described above.

[0045] Compared with the prior art, the embodiments of the present invention have at least the following advantages:

[0046] In this embodiment of the invention, in order to quickly and accurately respond to the real-time indicator requirements of online business systems or third-party business systems, business data is collected in real time through a client or terminal, and the collected business data is stored in a corresponding message queue. The data in the message queue is processed and calculated in real time, and the calculated data feature values ​​for the corresponding time period are stored in an intermediate storage container that supports efficient access. This allows the system to calculate the final data value according to the data feature values ​​corresponding to the time window when it receives a request to access data values ​​from an online business system or third-party business system, and synchronously return the calculated data value to the corresponding online business system or third-party business system. This provides efficient access support for the online business system or third-party business system and improves the real-time calculation accuracy of the online business system or third-party business system.

[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0048] Figure 1 This is a flowchart of a business data processing method provided in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram illustrating an application example of a business data processing method provided in an embodiment of the present invention;

[0050] Figure 3 This is another flowchart of a business data processing method provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram illustrating another application example of a business data processing method provided in an embodiment of the present invention;

[0052] Figure 5 This is a flowchart of a business data display method provided in an embodiment of the present invention;

[0053] Figure 6 This is a schematic diagram of the structure of a business data processing device provided in an embodiment of the present invention;

[0054] Figure 7 This is another structural schematic diagram of a business data processing device provided in an embodiment of the present invention;

[0055] Figure 8 This is a schematic diagram of the structure of a business data display device provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Before understanding this invention, let's introduce the technical terms related to this invention, as follows:

[0058] Real-time computing: refers to real-time streaming computing, which has different definitions in different fields. In this invention, it refers to real-time streaming computing in the field of big data. Real-time streaming computing can also be called real-time computing or streaming computing, etc., and the concepts are similar in the field of big data.

[0059] Time window: refers to a unit of time period, such as the most recent hour, 0:00 to 3:00 of the day, or 9:00 to 10:00, 8:10 to 10:00, etc. The length of the time period can be set as needed.

[0060] Sliding step size: The length of the sliding window, the time interval between window operations. A window operation is performed once every batch processing time interval.

[0061] Message queues, also known as message middleware, are containers that store messages during transmission. Common message queues include open-source components such as Kafka, RabbitMQ, and MetaQ.

[0062] For a technical explanation of the above technical terms, please refer to [link / reference]. Figure 1 The flowchart below illustrates a business data processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0063] Step 101: Obtain business data in real time;

[0064] Step 102: Store the business data in the corresponding message queue;

[0065] Step 103: Process the business data in each message queue according to the time dimension to obtain the data feature values ​​for the corresponding time period;

[0066] Step 104: Send the data feature value within the corresponding time period to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container can calculate the data value within the corresponding time window based on the data feature value and feed the data value back to the online business system or the third-party business system.

[0067] The business data processing method provided in this embodiment of the invention can be applied to mobile terminals, servers, clients, backend or online business systems or third-party business systems, etc., without limitation. The implementing device can be an electronic device such as a smartphone, laptop, or tablet computer, without limitation.

[0068] The following is combined with Figure 1 The specific implementation steps of the business data processing method provided in the embodiments of the present invention will be described in detail.

[0069] First, execute step 101 to obtain business data in real time.

[0070] The business data in this step can be user behavior data collected in real time by the backend, such as the number of pages viewed by a user. Alternatively, open-source real-time frameworks such as Spark Streaming, Flink, and Storm can be used to monitor user behavior data in real time.

[0071] Next, step 102 is executed to store the business data in the corresponding message queue.

[0072] In this step, real-time collected business data is stored in corresponding message queues. For example, subscribed messages are stored in the corresponding subscription message queue, and consumed messages are stored in the corresponding consumption message queue. Each message queue can consist of a Kafka instance. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data from consumers on a website. This includes data such as web browsing, searching, and other user action data. This behavioral data is typically handled through log processing and log aggregation due to throughput requirements. Data stored in Kafka occurs only sequentially, and the data deletion policy involves deleting data when it accumulates to a certain level or exceeds a certain time limit.

[0073] The purpose of Kafka message queues is to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time messaging through a cluster. Kafka message queues are used to store real-time business data information obtained from consumers.

[0074] Next, step 103 is executed, and the business data in each message queue is processed according to the time dimension to obtain the data feature values ​​for the corresponding time period.

[0075] In this step, the time dimension refers to using time as a metric to describe and express variables. That is, using time units to represent user behavior data within a given time frame (year, month, day, hour, minute, second, etc.). It's important to note that the time dimension itself has two dimensions: one is hierarchy (along the vertical direction), and the other is time (along the horizontal direction).

[0076] In this step, the business data in each message queue needs to be parsed and processed, and the dimensions and metrics to be calculated are extracted according to the time dimension to obtain the data feature values ​​for the current time period. For example, the data in the topic of the subscribed message queue or the consumer queue is parsed and processed in real time, and the data corresponding to the dimensions (i.e., identifiers used to represent user identity, etc.) and metrics (e.g., user IP, etc.) to be calculated are extracted from the raw data to obtain the data feature values ​​for the corresponding time period. Then, step 104 is executed to send the extracted data feature values ​​of dimensions and metrics to the first intermediate storage container.

[0077] Finally, step 104 is executed, in which the data feature values ​​within the corresponding time period are sent to the first intermediate storage container for storage. This allows the first intermediate storage container to calculate the data value within the corresponding time window based on the data feature values ​​when it receives a request from the online business system or a third-party business system to access the data value, and then feed the data value back to the online business system or the third-party business system.

[0078] In this step, the first intermediate storage container can be composed of Redis, or simply referred to as the first Redis. Redis is an open-source, network-enabled, in-memory, optional persistent high-performance key-value store database written in ANSI C; or, in other words, Redis is an open-source, in-memory data structure server that can be used as a database, cache, and message queue broker. It supports data types such as strings, hash tables, lists, sets, sorted sets, bitmaps, and hyperloglogs. This Redis can serve as an efficient intermediate storage container, providing high-efficiency access support for online business systems or third-party business systems.

[0079] In this embodiment, the first Redis includes a Zset (also known as a Redis Zset collection or Redis Zset structure). The Zset structure stores the data feature values ​​received within the corresponding time period in chronological order. For example, the data feature values ​​stored in the Zset structure within the corresponding time period are as follows: time-1, corresponding data feature value is k-1; time-2, corresponding data feature value is k-2; time-3, corresponding data feature value is k-3, and so on.

[0080] When the first Redis receives a request from an online business system or a third-party business system to access data values, the first Redis calculates the data value within the corresponding time window based on the data feature value and feeds the data value back to the online business system or the third-party business system, providing efficient access support for the online business system or the third-party business system.

[0081] In this embodiment of the invention, in order to quickly and accurately respond to the real-time indicator requirements of online business systems or third-party business systems, business data is collected in real time through a client or terminal, and the collected business data is stored in a corresponding message queue. The data in the message queue is processed and calculated in real time, and the calculated data feature values ​​for the corresponding time period are stored in an intermediate storage container that supports efficient access. This allows the system to calculate the final data value according to the data feature values ​​corresponding to the time window when it receives a request to access data values ​​from an online business system or third-party business system, and synchronously return the calculated data value to the corresponding online business system or third-party business system. This provides efficient access support for the online business system or third-party business system and improves the real-time calculation accuracy of the online business system or third-party business system.

[0082] For better understanding, please also refer to [the relevant documents / references]. Figure 2 This is a schematic diagram illustrating an application example of a business data processing method provided in an embodiment of the present invention, such as... Figure 2 As shown, suppose the message queue stores three data points: k-3, k-4, and k-5 (using three data points as an example; in practical applications, this is not the only possibility). The three data points in the message queue are parsed sequentially according to the time dimension to obtain the data header feature values ​​for the corresponding time periods, such as k-1, k-2, and k-3. Then, the data feature value corresponding to time-1 is k-1 (the correspondence between them can be represented as time-1 to k-1, or a key-value pair); the data feature value corresponding to time-2 is k-2 (the correspondence between them can be represented as time-2 to k-2); and the data feature value corresponding to time-3 is k-3 (the correspondence between them can be represented as time-3 to k-3). These are stored sequentially in a Redis Zset structure according to time. Zset is an upgraded version of set, adding an order attribute to the set. A set is a collection of unique values. Using the Redis Set data structure, you can store collection-like data.

[0083] When Redis receives a request to access data values ​​from other online or third-party business systems, it calculates the final data value according to the data feature value corresponding to the time window and synchronously returns the calculated data value to the corresponding online or third-party business system. This provides efficient access support for the online or third-party business systems and improves the real-time calculation accuracy of the online or third-party business systems.

[0084] It should be noted that the business data processing method provided in the above embodiments is applicable to situations where the online access volume is not very large. If the online business access volume is not high, the quantity is large, and the time window period is long, it will cause Redis to be overloaded and the response efficiency to be low. Based on this, please refer to the optimization scheme below.

[0085] Please also see Figure 3 This is another flowchart of a business data processing method provided in an embodiment of the present invention, the method comprising:

[0086] Step 301: Obtain business data in real time;

[0087] Step 302: Store the business data in the corresponding message queue;

[0088] Step 303: Process the business data in each message queue according to the time dimension to obtain the data feature values ​​for the corresponding time period;

[0089] It should be noted that steps 301 to 303 are the same as steps 101 to 103. For details of their implementation, please refer to the corresponding steps in the above embodiments, which will not be repeated here.

[0090] Step 304: Accumulate and / or deduplicate the data feature values ​​within the corresponding time period;

[0091] In this step, one scenario involves performing an accumulation calculation based on user requirements. This means sending the data feature values ​​within the corresponding time period to a first delayed queue composed of a Kafka structure. The first delayed queue stores the data feature values ​​according to their corresponding timestamps and then notifies the first computing center to perform the accumulation calculation on the data feature values. Additionally, the first delayed queue manages the expiration of the received data feature values ​​and sends expired data feature values ​​back to the first computing center, which then subtracts the expired data feature values ​​from the accumulation calculation result. It should be noted that the function of the first delayed queue is to immediately send the received data (i.e., data feature values) to the first computing center, which then simultaneously resends it according to the time delay of the calculation time window.

[0092] For example, counting website downloads doesn't require stored procedures to reduce resource overhead; it only requires accumulating the count over a time window. Therefore, a delayed queue approach can be used. Data within each time period is delayed for a specified time before being resent back to the computing center. When new data is received, it's summed to the count using an accumulator, and simultaneously placed in the delayed queue for a window period. When the computing center receives data from the delayed queue, it sums it or accumulates the corresponding negative value using an accumulator.

[0093] Another scenario involves deduplication based on user requirements. This involves sending the data feature values ​​within the corresponding time period to a sequential queue composed of an intermediate storage container set (e.g., a combination of Redis Zset and a similar structure) and a second delayed queue composed of a Kafka structure. The sequential queue and the second delayed queue store the data feature values ​​according to their corresponding timestamps. The sequential queue prunes the stored data feature values ​​according to a window size and, after pruning, notifies the second computing center to calculate the number of remaining data feature values ​​in the current sequential queue. Furthermore, when the sequential queue receives an expired message from the second delayed queue, it prunes the data according to the window size based on the expired message and, after pruning, notifies the second computing center to calculate the number of remaining K values ​​in the current queue.

[0094] For example, calculations requiring duplicate value removal, such as counting UV, maximum value, and minimum value, can be performed using a combination of a delayed queue (i.e., a second delayed queue, to distinguish it from the first delayed queue mentioned above) and a sequential queue. The second delayed queue functions identically to the first delayed queue mentioned above. The difference in this embodiment is to reduce the resource overhead of the delayed queue; it does not need to store detailed data, only a signal. The sequential queue is implemented using a Redis Zset structure, maintaining a detailed record of received data. The queue follows a first-in, first-out (FIFO) principle, disallowing queue jumping. When new data is received and needs to be added to the queue, the new data is placed at the end of the sequential queue, with a timestamp used as the index of the current data. Alternatively, when the delayed queue sends a signal, expired data at the head of the queue is checked and dequeued, while the second computing center is notified to recalculate the data value.

[0095] Another scenario involves performing cumulative calculations and deduplication calculations simultaneously based on different user needs. This means sending the data feature values ​​for one user within a given time period to the first delay queue and the first computing center, and sending the data feature values ​​for another user within a given time period to the sequential queue and the second delay queue. The subsequent processing procedures are detailed in the corresponding methods described above and will not be repeated here.

[0096] Step 305: Send the results of the cumulative calculation and / or deduplication calculation to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data values ​​within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed back the data values ​​to the online business system or the third-party business system.

[0097] In this step, the results of the cumulative calculation and / or deduplication calculation are sent to a second intermediate storage container for storage. This second intermediate storage container can also be composed of Redis. In other words, the result obtained through cumulative calculation and / or deduplication is the accumulated data value or the deduplicated data value, for example, K:2. The second intermediate storage container stores the data value K:2. Therefore, this type of intermediate storage container can support a large volume of online business access or browsing access with a long time window. That is, Redis acts as an intermediate storage container for online business systems or third-party business systems, providing efficient access support for these systems.

[0098] In this embodiment of the invention, in order to quickly and accurately respond to the real-time indicator requirements of online business systems or third-party business systems, business data is collected in real time through a client or terminal, and the collected business data is stored in a corresponding message queue. The data in the message queue is processed and calculated in real time. The calculated data feature values ​​within the corresponding time period are accumulated and / or deduplicated. The results of the accumulation and / or deduplication calculations are stored in an intermediate storage container that supports efficient access. This allows the system to calculate the final data value according to the data feature values ​​corresponding to the time window when it receives a request for access data value from an online business system or third-party business system. The calculated data value is then synchronously returned to the corresponding online business system or third-party business system, thereby providing efficient access support for the online business system or third-party business system and improving the real-time calculation accuracy of the online business system or third-party business system.

[0099] For better understanding, please also refer to [the relevant documents / references]. Figure 4 This is a schematic diagram illustrating another application example of a business data processing method provided in an embodiment of the present invention, such as... Figure 4 As shown, suppose the message queue stores three data points k-3, k-4, and k-5 (using three data points as an example, but not limited to this in actual applications). The three data points in the message queue are processed sequentially according to the time dimension (such as parsing), and the data header feature values ​​for the corresponding time period are obtained, such as k-1, k-2, and k-3. Then, k-1, k-2, and k-3 are sent to the corresponding delayed queue and sequential queue for processing, respectively.

[0100] In this embodiment, for the cumulative calculation, the data feature value corresponding to the time obtained after data processing is sent to the corresponding first delay queue (i.e., delay queue 1, which is composed of a Kafka structure). The first delay queue stores the data and its corresponding timestamp at the tail of the queue. Then, the first delay queue notifies the first computing center (i.e., the computing center for the cumulative calculation) to perform the cumulative calculation on the data feature value, that is, the first computing center increments the accumulated data value by 1. When the first delay queue detects that there is expired or outdated data in its stored data feature value, such as detecting that offset-1 has expired, the first delay queue will send the expired or outdated data feature value (e.g., offset-1) to the first computing center. The first computing center then dequeues the expired or outdated data feature value, that is, decrements the accumulated data value by 1, thus achieving the purpose of clearing expired or outdated data.

[0101] It should be noted that, for the cumulative calculation provided in the embodiments of the present invention, the delayed queue can notify the first computing center of the expired data, and the first computing center can perform the corresponding subtraction, so that the calculation window period slides forward according to the event, without the need to store the detailed data within the window period, which effectively improves the accuracy of data value calculation within the time window.

[0102] For the deduplication calculation, the data feature values ​​obtained after data processing are sent to the second delayed queue (i.e., delayed queue 2, which is composed of Kafka structures) and the sequential queue (which is composed of Redis Zset sets or structures).

[0103] In this embodiment, the sequential queue is implemented based on the Redis Zset (value, score) structure, using the timestamp as the score. Zset has deduplication functionality, storing the same K value only once. The score is the timestamp of the last storage, and the data feature values ​​in the sequential queue are arranged from beginning to end in chronological order.

[0104] In this embodiment, the pruning method used in the sequential queue is as follows: the data feature values ​​in the sequential queue are arranged from beginning to end in chronological order. Therefore, the data feature values ​​at the head of the sequential queue are earlier data feature values. For example, if the window period is 1 hour, then based on the current time, if there is data in the sequential queue that is more than 1 hour earlier, the sequential queue will prune from the time point 1 hour earlier to the head of the queue. Therefore, after pruning, the sequential queue will only store the data feature values ​​of the most recent 1 hour after removing duplicates.

[0105] For example, in this embodiment, when the sequential queue receives new data K-3, it places (data K-3, timestamp time-3) at the end of the sequential queue. Then, the sequential queue is pruned from the head according to the window size. After the pruning is completed, the deduplication calculation center (i.e., the second calculation center) is notified to calculate the number of remaining K values ​​in the current sequential queue.

[0106] When the sequential queue receives a message from the delayed queue, it will directly prune the head of the queue according to the window size. After the pruning is completed, it will notify the second computing center to calculate the number of remaining K values ​​in the current queue.

[0107] It should be noted that the deduplication calculation provided in this embodiment of the invention, through the sequential queue pruning implemented in Redis, can accurately calculate the value within the event-based sliding window period. Compared with common approximate deduplication algorithms (HyperLogLog, which can save storage space but cannot solve the problem of data calculation accuracy and the problem of expired data being eliminated when the window period slides forward based on events), this embodiment of the invention not only saves storage space but also improves the accuracy of data volume calculation.

[0108] Regarding the deduplication calculation provided in the embodiments of the present invention, the API provided by Flink technology can currently support event-triggered sliding window calculation. However, when a large window period occurs, the event-triggered forward sliding calculation still needs to adopt the embodiments provided in the present invention.

[0109] Finally, the results of the cumulative calculation and / or deduplication calculation are sent to a second intermediate storage container, Redis, for storage. This allows the second Redis to calculate the data value within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation when it receives a request from an online business system to access data values, and then feed that data value back to the online business system or third-party business system. This provides efficient access support for the online business system or third-party business system and improves the real-time calculation accuracy of the online business system or third-party business system.

[0110] In this embodiment of the invention, real-time collected business data is processed and calculated. The calculated data feature values ​​for the corresponding time period are stored in a delay queue and a sequential queue. The corresponding computing centers perform cumulative calculations and / or deduplication calculations, and the results of the cumulative calculations and / or deduplication calculations are stored in an intermediate storage container, Redis. This allows the system to calculate the final data value according to the data feature values ​​corresponding to the time window when it receives a request to access data values ​​from an online business system or a third-party business system. The calculated data value is then synchronously returned to the corresponding online business system or third-party business system, thereby providing efficient access support for the online business system or third-party business system and improving the real-time calculation accuracy of the online business system or third-party business system.

[0111] Please also see Figure 5 The flowchart illustrates a business data display method provided in an embodiment of the present invention, the method comprising:

[0112] Step 501: Receive a query request from a user regarding their online business operations;

[0113] In this step, the online business system or a third-party business system receives a query request from a user regarding their online business operations. This query request may include the identifier of the user to be queried.

[0114] The online business system or third-party business system can be a search, recommendation, risk control, artificial intelligence, or other systems that use the calculation methods of this invention to query and browse the traffic of various Internet products.

[0115] For example, if an online business system discovers that a user is maliciously using a web crawler to obtain a large amount of information from a website, it needs to identify and block such malicious crawlers. For this type of business scenario, it is usually necessary to collect user behavior data in real time and calculate the number of pages viewed by the user. In this case, the administrator sends a query request for that user through the online business system.

[0116] Step 502: Based on the query request, request the data value of the current user's online business operation from the intermediate storage container; so that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation in the current time window based on the data feature value stored for each time period, and return the data value.

[0117] In this step, the intermediate storage container can be either a first intermediate storage container or a second intermediate storage container. The functions and roles of the first and second intermediate storage containers are detailed above and will not be repeated here.

[0118] In this step, the online business system or third-party business system requests the data value of the current user's online business operation from the intermediate storage container (such as Redis) according to the query request. Upon receiving the query request, Redis calculates the data value of the user's online business operation within the current time window based on the data feature values ​​stored for each time period, and then feeds back the data value.

[0119] The process by which Redis calculates the data value of the user's online business operation within the current time window based on the data feature values ​​stored for each time period upon receiving the query request is detailed in the corresponding embodiments described above and will not be repeated here.

[0120] Step 503: Receive the data value of the user's online service operation within the current time window sent by the intermediate storage container;

[0121] In this step, the online business system or third-party business system receives the data value of the user's online business operation within the current time window from the intermediate storage container.

[0122] Step 504: Display the data value of the user's online business operation so as to determine whether to process the user's online business operation based on the data value.

[0123] In this step, the online business system or third-party business system displays the user's data value to the administrator so that the administrator can determine whether to process the user's online business operations based on the data value. For example, if the administrator determines based on the data value that the same user has accessed more than 500 pages in 1 hour, then the access volume of the user needs to be blocked in time.

[0124] In this embodiment of the invention, when an online business system or a third-party business system receives a query request for a user's online business operations, it requests the data value of the current user's online business operations from the intermediate storage container Redis according to the query request, and receives the data value of the user's online business operations within the current time window from Redis; then, it displays the data value of the user's online business operations. In other words, in this embodiment of the invention, when the online business system or a third-party business system receives a query request, it can quickly obtain the data value of the user's online business operations within the current time window from Redis and display it to the administrator, so that the administrator can process the user's online business operations based on the data value. That is, the technical solution provided by this embodiment of the invention can solve the problem of computational errors caused by the mismatch between computation time and event time when performing real-time calculations using conventional computing framework components. This not only saves time in querying user online business operations but also improves the accuracy of calculating user online business operations within the current time window.

[0125] It should be noted that, in this embodiment of the invention, since various Internet products such as search, recommendation, risk control, and artificial intelligence require a large amount of real-time indicator calculation time window traffic to support upper-layer applications, the accuracy, real-time performance, and concurrent response speed of the calculation determine the quality of the Internet product.

[0126] Especially in internet products, such as search and recommendation systems, a large amount of data feature values ​​are needed to characterize user behavior in order to uncover potential user needs, shorten the distance between users and products or information, and improve the user experience. Similarly, in the field of information security, strategy engines built on artificial intelligence technology have penetrated all aspects of risk control product functions. Correspondingly, each strategy system relies on a large amount of feature calculation to support the accurate response of model algorithms or manual rules to requests. Therefore, the calculation speed, response time, and accuracy of the data feature values ​​provided in this embodiment directly affect the judgment of user needs by these online systems.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0128] Please see Figure 6This is a schematic diagram of the structure of a business data processing device provided in an embodiment of the present invention. Specifically, it may include the following modules: an acquisition module 601, a first sending module 602, a processing module 603, and a second sending module 604, wherein...

[0129] The acquisition module 601 is used to acquire business data in real time;

[0130] The first sending module 602 is used to send the service data to the corresponding message queue;

[0131] The processing module 603 is used to process the business data in each message queue according to the time dimension to obtain the data feature value within the corresponding time period.

[0132] The second sending module 604 is used to send the data feature value obtained by the processing module within a corresponding time period to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container calculates the data value within the corresponding time window based on the data feature value and feeds back the data value to the online business system or the third-party business system.

[0133] Optionally, the device further includes a calculation module 701 and a third transmission module 702, the structural diagram of which is shown below. Figure 7 As shown, where,

[0134] The calculation module 701 is used to perform cumulative calculation and / or deduplication calculation on the data feature values ​​within the corresponding time period after the processing module 603 obtains the data feature values ​​within the corresponding time period.

[0135] The third sending module 702 is used to send the results of the cumulative calculation and / or deduplication calculation calculated by the calculation module 701 to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data value within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed the data value back to the online business system or the third-party business system.

[0136] Optionally, the calculation module may include: a fourth sending module and an accumulation calculation module; or it may include: a fifth sending module and a deduplication calculation module; or it may include: a fourth sending module, an accumulation calculation module, a fifth sending module, and a deduplication calculation module, wherein,

[0137] The fourth sending module is used to send the data feature values ​​within the corresponding time period to the first delay queue composed of a Kafka structure, and the first delay queue stores the data feature values ​​according to the corresponding timestamps and then notifies the first computing center.

[0138] The cumulative calculation module is used to perform cumulative calculation on the data feature value through the first computing center, and to perform expiration management on the received data feature value through the first delay queue, and send the expired data feature value to the first computing center, so that the first computing center subtracts the expired data feature value from the result of the cumulative calculation;

[0139] The fifth sending module is used to send the data feature values ​​within the corresponding time period to a sequential queue composed of an intermediate storage container set and a second delayed queue composed of Kafka containers, respectively; the sequential queue and the second delayed queue store the data feature values ​​according to the corresponding timestamps, respectively.

[0140] The deduplication module is used to prune the stored data feature values ​​according to the window size through the sequential queue, and after the pruning is completed, notify the second computing center to calculate the number of remaining data feature values ​​in the current sequential queue; and when the sequential queue receives an expired message sent by the second delayed queue, it prunes the data according to the window size based on the expired message, and after the pruning is completed, notifies the second computing center to calculate the number of remaining data feature values ​​in the current queue.

[0141] Please see Figure 8 This is a schematic diagram of a business data display device provided in an embodiment of the present invention. The device includes: a first receiving module 801, a request module 802, a second receiving module 803, and a display module 804, wherein...

[0142] The first receiving module 801 is used to receive query requests from users regarding their online business operations.

[0143] The request module 802 is used to request the data value of the current user's online business operation from the intermediate storage container according to the query request; so that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation in the current time window according to the data feature value stored in each time period, and feed back the data value.

[0144] The second receiving module 803 is used to receive the data value of the user's online service operation within the current time window from the intermediate storage container;

[0145] The display module 804 is used to display the data values ​​of the user's online business operations, so as to determine whether to process the user's online business operations based on the data values.

[0146] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0147] Optionally, embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the business data processing method described above or implements the various processes of the business data display method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0148] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the steps of the business data processing method described above or the various processes of the business data display method embodiments described above, achieving the same technical effects. To avoid repetition, further details are omitted here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0149] In an exemplary embodiment, a computer program product is also provided. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the steps of the business data processing method described above or implements the various processes of the business data display method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0150] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0151] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminals (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0155] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0156] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0157] The above provides a detailed description of the business data processing method, business data display method, apparatus, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A business data processing method, characterized in that, include: Real-time acquisition of business data; The business data is stored in the corresponding message queue; The business data in each message queue is processed according to the time dimension to obtain the data feature value for the corresponding time period. The data feature value is obtained by extracting dimensions and indicators from the business data according to the time dimension and performing calculation processing. The dimension is used to represent the identifier of the user's identity, and the indicator includes the user's IP. The data feature value within the corresponding time period is sent to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container calculates the data value within the corresponding time window based on the data feature value and feeds the data value back to the online business system or the third-party business system. The first intermediate storage container combines the data feature value and the time of the data feature value into a key-value pair for storage. The method further includes: The data feature values ​​within the corresponding time period are stored in a delayed queue and a sequential queue, and are accumulated and / or deduplicated by the corresponding computing centers respectively; wherein, the delayed queue is used to notify the corresponding computing center of the expired data feature values; the delayed queue is composed of a Kafka structure; the sequential queue is used to prune the stored data feature values ​​according to the window size, or when receiving the expiration notification sent by the delayed queue, prune the stored data feature values ​​according to the window size based on the expiration message, and after the pruning is completed, notify the corresponding computing center to calculate the number of remaining data feature values ​​in the sequential queue; The results of the cumulative calculation and / or deduplication calculation are sent to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data values ​​within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed back the data values ​​to the online business system or the third-party business system.

2. The method according to claim 1, characterized in that, After processing the business data in each message queue according to the time dimension to obtain the data feature values ​​for the corresponding time period, the method further includes: The data feature values ​​within the corresponding time period are accumulated and / or deduplicated.

3. The method according to claim 2, characterized in that, The step of accumulating and / or deduplicating the data feature values ​​within the corresponding time period includes: The data feature values ​​within the corresponding time period are sent to a first delay queue composed of a Kafka structure. The first delay queue stores the data feature values ​​according to the corresponding timestamps and then notifies the first computing center to perform cumulative calculation on the data feature values. The first delay queue also manages the expiration of the received data feature values ​​and sends the expired data feature values ​​to the first computing center, which then subtracts the expired data feature values ​​from the cumulative calculation result. and / or The data feature values ​​within the corresponding time period are sent to a sequential queue composed of an intermediate storage container set and a second delayed queue composed of Kafka containers, respectively. The sequential queue and the second delayed queue store the data feature values ​​according to their corresponding timestamps. The sequential queue prunes the stored data feature values ​​according to the window size and notifies the second computing center after pruning to calculate the number of remaining data feature values ​​in the current sequential queue. When the sequential queue receives an expired message from the second delayed queue, it prunes the data according to the window size based on the expired message and notifies the second computing center after pruning to calculate the number of remaining data feature values ​​in the current queue.

4. A method for displaying business data, characterized in that, include: Received a query request from a user regarding their online business operations; Based on the query request, request the data value of the current user's online business operation from the intermediate storage container; So that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation in the current time window based on the data feature value stored in each time period, and feed back the data value. The data feature value is obtained by extracting dimensions and indicators from the business data according to the time dimension and performing calculation processing. The dimension is used to represent the identifier of the user's identity, and the indicator includes the user's IP. The intermediate storage container receives the data value of the user's online service operation within the current time window, and stores the data feature value and the time of the data feature value as a key-value pair. Display the data values ​​of the user's online business operations so as to determine whether to process the user's online business operations based on the data values; The method further includes: The data value is obtained by storing the data feature values ​​within the corresponding time period in a delayed queue and a sequential queue, and performing cumulative calculations and / or deduplication calculations through the corresponding computing centers respectively; wherein, the delayed queue is used to notify the corresponding computing center of the expired data feature values; the delayed queue is composed of a Kafka structure; the sequential queue is used to prune the stored data feature values ​​according to the window size, or when receiving the expiration notification sent by the delayed queue, prune the stored data feature values ​​according to the window size based on the expiration message, and after the pruning is completed, notify the corresponding computing center to calculate the number of remaining data feature values ​​in the sequential queue.

5. A business data processing device, characterized in that, include: The acquisition module is used to acquire business data in real time. The first sending module is used to send the business data to the corresponding message queue; The processing module is used to process the business data in each message queue according to the time dimension to obtain the data feature value for the corresponding time period. The data feature value is obtained by extracting dimensions and indicators from the business data according to the time dimension and performing calculation processing. The dimension is used to represent the identifier of the user's identity, and the indicator includes the user's IP. The second sending module is used to send the data feature value obtained by the processing module within a corresponding time period to the first intermediate storage container for storage, so that when the first intermediate storage container receives a request from the online business system or the third-party business system to access the data value, the first intermediate storage container calculates the data value within the corresponding time window based on the data feature value and feeds the data value back to the online business system or the third-party business system. The first intermediate storage container stores the data feature value and the time of the data feature value as a key-value pair. The device further includes: The data feature values ​​within the corresponding time period are stored in a delayed queue and a sequential queue, and are accumulated and / or deduplicated by the corresponding computing centers respectively; wherein, the delayed queue is used to notify the corresponding computing center of the expired data feature values; the delayed queue is composed of a Kafka structure; the sequential queue is used to prune the stored data feature values ​​according to the window size, or when receiving the expiration notification sent by the delayed queue, prune the stored data feature values ​​according to the window size based on the expiration message, and after the pruning is completed, notify the corresponding computing center to calculate the number of remaining data feature values ​​in the sequential queue; The third sending module is used to send the results of the cumulative calculation and / or deduplication calculation calculated by the calculation module to the second intermediate storage container for storage, so that when the second intermediate storage container receives a request from the online business system to access data values, it can calculate the data value within the corresponding time window based on the results of the cumulative calculation and / or deduplication calculation, and feed the data value back to the online business system or the third-party business system.

6. The apparatus according to claim 5, characterized in that, The device further includes: The calculation module is used to perform cumulative calculation and / or deduplication calculation on the data feature values ​​within the corresponding time period after the processing module obtains the data feature values ​​within the corresponding time period.

7. The apparatus according to claim 6, characterized in that, The calculation module includes: a fourth sending module and an accumulation calculation module; and / or, a fifth sending module and a deduplication calculation module, wherein, The fourth sending module is used to send the data feature values ​​within the corresponding time period to the first delay queue composed of a Kafka structure, and the first delay queue stores the data feature values ​​according to the corresponding timestamps and then notifies the first computing center. The cumulative calculation module is used to perform cumulative calculation on the data feature value through the first computing center, and to perform expiration management on the received data feature value through the first delay queue, and send the expired data feature value to the first computing center, so that the first computing center subtracts the expired data feature value from the result of the cumulative calculation. The fifth sending module is used to send the data feature values ​​within the corresponding time period to a sequential queue composed of an intermediate memory set and a second delayed queue composed of Kafka; the sequential queue and the second delayed queue store the data feature values ​​according to the corresponding timestamps. The deduplication calculation module is used to prune the stored data feature values ​​according to the window size through the sequential queue, and after the pruning is completed, notify the second computing center to calculate the number of remaining data feature values ​​in the current sequential queue; and when the sequential queue receives an expired message sent by the second delayed queue, it prunes the data according to the window size based on the expired message, and after the pruning is completed, notifies the second computing center to calculate the number of remaining data feature values ​​in the current queue.

8. A business data display device, characterized in that, include: The first receiving module is used to receive query requests from users regarding their online business operations. The request module is used to request the data value of the current user's online business operation from the intermediate storage container according to the query request; so that when the intermediate storage container receives the query request, it can calculate the data value of the user's online business operation in the current time window according to the data feature value stored for each time period, and return the data value. The data feature value is obtained by extracting dimensions and indicators from the business data according to the time dimension and performing calculation processing. The dimension is used to represent the user's identity identifier, and the indicator includes the user's IP address. The second receiving module is used to receive the data value of the user's online business operation within the current time window from the intermediate storage container. The intermediate storage container stores the data feature value and the time of the data feature value as a key-value pair. The display module is used to display the data values ​​of the user's online business operations, so as to determine whether to process the user's online business operations based on the data values; The device further includes: The data value is obtained by storing the data feature values ​​within the corresponding time period in a delayed queue and a sequential queue, and performing cumulative calculations and / or deduplication calculations through the corresponding computing centers respectively; wherein, the delayed queue is used to notify the corresponding computing center of the expired data feature values; the delayed queue is composed of a Kafka structure; the sequential queue is used to prune the stored data feature values ​​according to the window size, or when receiving the expiration notification sent by the delayed queue, prune the stored data feature values ​​according to the window size based on the expiration message, and after the pruning is completed, notify the corresponding computing center to calculate the number of remaining data feature values ​​in the sequential queue.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the business data processing method as described in any one of claims 1 to 3 or the steps of the business data display method as described in claim 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the business data processing method as described in any one of claims 1 to 3 or the steps of the business data display method as described in claim 4.

Images