Method for processing streaming data, electronic device and computer program product

Abstract

This disclosure provides a method for processing streaming data, an electronic device, and a computer program product. The method of this disclosure first determines the metadata of the streaming data in response to receiving streaming data from a user. The metadata includes a user ID and a timestamp of the streaming data. Then, the target time window to which the streaming data belongs is determined by the user ID and the timestamp. Next, the statistical information of the target time window is updated. The statistical information includes the statistical results of at least one window indicator of the target time window. Finally, if the update of the statistical information causes an event triggering condition to be met, the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

Description

Technical Field

[0001] This disclosure specifically relates to methods for processing streaming data, electronic devices, readable storage media, and computer program products. Background Technology

[0002] In streaming data processing scenarios, business users need to acquire the data generated when the executing entity performs certain operations, perform statistics and summarize this data, and trigger downstream processing logic when the data statistics results meet specific rule conditions, so as to use and apply this data.

[0003] The common practice is to configure time windows by setting a TTL (Time-to-Live) and determine the time window to which the streaming data belongs based on the time it takes for the system to receive the streaming data. If the streaming data is delayed and arrives at the system after the TTL has expired, the corresponding time window will be closed, resulting in the streaming data not being counted correctly. This leads to inaccurate statistical results and deviations in the processing results of subsequent downstream processing logic. Summary of the Invention

[0004] This disclosure provides methods for processing streaming data, electronic devices, readable storage media, and computer program products.

[0005] The first aspect of this disclosure proposes a method for processing streaming data, characterized by comprising: in response to receiving streaming data from a user, determining metadata of the streaming data, the metadata including a user ID and a timestamp of the streaming data; determining a target time window to which the streaming data belongs based on the user ID and the timestamp; updating statistical information of the target time window, the statistical information including statistical results of at least one window indicator of the target time window; and if the update of the statistical information causes an event triggering condition to be met, triggering a processing flow for the streaming data belonging to the target time window based on the latest statistical information of the target time window.

[0006] According to some embodiments of this disclosure, determining the target time window to which the streaming data belongs by means of the user ID and the timestamp includes: if no streaming data belonging to the user ID has been received before the streaming data is received, then opening a time window corresponding to the user ID and using it as the target time window to which the streaming data belongs.

[0007] According to some embodiments of this disclosure, after opening a time window corresponding to the user ID and using it as the target time window to which the streaming data belongs, the method further includes: using the timestamp as the reference time of the target time window, wherein the reference time is used to determine whether the time window is in a closed state.

[0008] According to some embodiments of this disclosure, determining the target time window to which the streaming data belongs by means of the user ID and the timestamp includes: if streaming data belonging to the user ID has been received before the streaming data is received, then comparing the timestamp with a reference time of the latest opened time window corresponding to the user ID, wherein the reference time is used to determine whether the time window is closed; and determining the target time window to which the streaming data belongs based on the comparison result.

[0009] According to some embodiments of this disclosure, determining the target time window to which the streaming data belongs based on the comparison result includes: if the timestamp is later than the reference time, and the latest opened time window is determined to be not closed by the reference time, then the latest opened time window is taken as the target time window to which the streaming data belongs.

[0010] According to some embodiments of this disclosure, determining the target time window to which the streaming data belongs based on the comparison result includes: if the timestamp is not later than the reference time, then opening a new time window corresponding to the user ID and using it as the target time window to which the streaming data belongs.

[0011] According to some embodiments of this disclosure, before the statistical information of the newly opened target time window is updated for the first time, at least some of the window indicators in the statistical information are set to default values.

[0012] According to some embodiments of this disclosure, if the event triggering condition is met, the reference time is also reset to the maximum allowed value, and the reference time being reset to the maximum value indicates that the corresponding time window is in a closed state.

[0013] According to some embodiments of this disclosure, updating the statistical information of the target time window includes: performing incremental aggregation calculations on the window indicators of the target time window through an aggregation engine to complete the update of the statistical information of the target time window.

[0014] According to some embodiments of this disclosure, the window metric includes: the number of times the target time window is allocated with the streaming data, and the event triggering condition includes: the number of allocations is not less than a number threshold.

[0015] According to some embodiments of this disclosure, the event triggering condition includes: the target time window is in an overdue state; after updating the statistical information of the target time window, the method further includes: if the working state of the target time window changes so that the event triggering condition is met, then the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

[0016] According to some embodiments of this disclosure, triggering a processing flow for streaming data belonging to the target time window based on the latest statistical information of the target time window includes: generating trigger event information containing a trigger identifier and the latest statistical information of the target time window, wherein the trigger identifier is used to indicate whether the event triggering condition is met; and triggering a processing flow for streaming data belonging to the target time window in response to the generation of the trigger event information.

[0017] According to some embodiments of this disclosure, at least one of the streaming data, the statistical information, and the triggering event information is stored in a storage space other than memory in the form of structured data.

[0018] According to some embodiments of this disclosure, when the statistical information of the newly opened target time window is updated for the first time, the value of the specified parameter is updated to a first value only if the value of the specified parameter is an initial value. The specified parameter is used to represent the data flow order. The action of resetting the base time can be triggered only if the value of the specified parameter is an initial value.

[0019] According to some embodiments of this disclosure, only when the value of the specified parameter is a first value, the action of using the timestamp as the reference time of the new time window and the action of determining whether the event triggering condition is met can be triggered; when the generation of the triggering event information is completed, the value of the specified parameter is updated to a second value, and only when the value of the specified parameter is a second value, the action of resetting the statistical results in the statistical information of the target time window and the action of updating the value of the specified parameter to the initial value can be triggered.

[0020] A second aspect of this disclosure provides an electronic device, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform the method described in any of the above embodiments.

[0021] A third aspect of this disclosure provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the method described in any of the above embodiments.

[0022] The fourth aspect of this disclosure provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the method described in any of the above embodiments. Attached Figure Description

[0023] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0024] Figure 1 The diagram illustrates an application scenario of the streaming data processing method according to some embodiments of this disclosure.

[0025] Figure 2 A schematic diagram of the overall flow of a streaming data processing method M100 according to some embodiments of the present disclosure is shown.

[0026] Figure 3 A schematic diagram of the overall flow of a streaming data processing method M100 in the case of receiving streaming data from user u1 for the first time, according to some embodiments of this disclosure, is shown.

[0027] Figures 4-5 A schematic diagram of the overall flow of a streaming data processing method M100 in the case of receiving streaming data from user u1 for the first time, according to some embodiments of this disclosure, is shown.

[0028] Figures 6-8 A schematic flowchart of a streaming data processing method M100 according to other embodiments of this disclosure is shown.

[0029] Figure 9 An overall flowchart of a method for processing streaming data according to some embodiments of the present disclosure is shown.

[0030] Figure 10 This is a schematic block diagram of a streaming data processing apparatus according to one embodiment of the present disclosure.

[0031] Figure 11 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure. Detailed Implementation

[0032] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0033] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways in which the technical concepts of this disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of this disclosure.

[0035] The terminology used herein is for the purpose of describing particular embodiments and is not restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0036] Streaming data processing refers to the technology of processing and analyzing continuously, high-speed, and real-time generated data in real time. It is applicable to scenarios such as device monitoring, user behavior analysis, and network security log analysis. Taking user behavior analysis as an example, business users need to parse user behavior (such as interactive text and call recordings) in real time to generate real-time tags, thereby quickly generating the latest decision support.

[0037] The parsing of interactive text and call recordings usually relies on large AI (artificial intelligence) models. Therefore, in order to improve the accuracy of tag generation and reduce the cost of calling AI, business parties usually only summarize and analyze users' historical behavior after the user's behavior meets certain rules. This rule can be: the user communicates with the business party's APP (application) N1 times within M1 minutes.

[0038] Currently, a common approach is to process streaming data using an architecture combining Kafka (a distributed stream processing platform) and Redis (a key-value store). When streaming data is sent to Kafka, user behavior statistics are performed based on Redis's Time-to-Live (TTL) feature. A fixed TTL is set as the lifespan of the time window, and the time window to which the streaming data belongs is allocated according to the time the streaming data is received. Here, a time window refers to a specific time interval within which certain operations, events, or data are allowed to be processed, occur, or become valid. In this case, the time window can specifically refer to a dynamic session window that starts at the time of the user's first action and lasts for a certain period.

[0039] For example, if the TTL (Time To Live) is configured to 10 minutes, time window s1 opens at time t1. At time t1+1 minutes, the Redis system receives streaming data k1 containing user interaction content, and then assigns streaming data k1 to time window s1. Afterwards, until time t1+10 minutes, the Redis system does not receive any new streaming data. Because the TTL has been reached, time window s1 is closed.

[0040] Due to communication latency, a data stream k2 is received by the Redis system at time t1+11min. In reality, data stream k2 should have been received before time window s1 closed. However, since time window s1 has already closed, data stream k2 will either be discarded or allocated to a new time window s2 following time window s1. This will lead to inaccurate statistical data in time window s1, resulting in statistical distortion and ultimately reducing the accuracy of the generated user tags.

[0041] Therefore, this disclosure proposes a method for processing streaming data.

[0042] Figure 1 The diagram illustrates an application scenario of a streaming data processing method according to some embodiments of this disclosure. In this application scenario, a user terminal 10 and a server terminal 20 may be included. The server terminal 20 is capable of communicating with the user terminal 10 to send and receive data or commands. In this disclosure, both the user terminal 10 and the server terminal 20 include at least one processor and at least one memory.

[0043] For example, client 10 may be a terminal device held by a user, which generates streaming data by operating an application installed on the terminal device. Server 20 may include a server that can obtain streaming data from each user through client 10, perform user behavior statistics on the streaming data of different users, and generate user tags based on the statistical results. The generated user tags are then used to construct and update user profiles that characterize user features and needs.

[0044] exist Figure 1 The shape and structure of the user terminal 10 and the server terminal 20 shown should not be construed as limiting the scope of protection of this disclosure. In this disclosure, "terminal device" can be different types of electronic devices, such as mobile phones, tablets, laptops, or desktop computers. Furthermore, the server can be a physical server or a cloud server; this disclosure does not limit the type of server.

[0045] Figure 2 A schematic flowchart of a streaming data processing method M100 according to some embodiments of this disclosure is shown. For example... Figure 2 The method shown includes steps S110, S120, S130, and S140. This method can be executed by electronic devices such as computers and servers.

[0046] S110, in response to receiving user streaming data, determine the metadata of the streaming data. The metadata includes the user ID and the timestamp of the streaming data.

[0047] When users interact with the business's application on their mobile phones or the business's platform page on their computers, they can perform actions such as conversing with the intelligent agent, clicking or browsing pages, searching for specific content, and sharing or commenting on specific content. The application or page can acquire these user behavior events and send the event stream data to the business's system (such as a server).

[0048] After receiving streaming data generated by a user, the system first determines the user ID and timestamp corresponding to the streaming data. The determined user ID is used to allocate the streaming data, ensuring that streaming data from different users is processed separately and not mixed together. The timestamp indicates the time when the streaming data was generated (the time when the user's action occurred). For example, for an event where a user engages in a conversation with an agent, the generation time of the streaming data could be the moment when the front-end (mobile phone or computer) running the application or platform page receives the conversation content input by the user. Metadata information of the streaming data is usually contained within the streaming data itself; it can be obtained by parsing the streaming data.

[0049] S120 determines the target time window to which the streaming data belongs by using the user ID and timestamp.

[0050] At the same time, multiple different users can generate their own streaming data in various ways, and the streaming data of different users is processed through different time windows. After receiving the streaming data, the system uses the user ID to determine the target time window corresponding to that user, and the timestamp is determined according to the generation time of the streaming data rather than the time the system receives it. In this way, even if there is a time difference between the arrival time of the streaming data and the actual generation time due to communication delays, the determined target time window can still be the correct time window to which the streaming data should belong.

[0051] For example, the system receives streaming data k1 from user u1 at time t1, thus opening time window s1. Then, it does not receive the next streaming data k2 from user u1 until time t1+20min. At this time, time window s1 is in an expired state, but the timestamp of streaming data k2 is time t1+2min. This timestamp is before time window s1 expires, indicating that if no communication delay occurs, data stream k2 should be determined to belong to time window s1. Therefore, the time window to which streaming data k2 belongs is determined by the timestamp of streaming data k2, thereby determining that time window s1 is the target time window for streaming data k2, rather than discarding streaming data k2 or assigning it to a new time window.

[0052] Understandably, the overdue state refers to the working state that the time window starts counting from the opening time and enters after the count value reaches the specified duration. When the count value has not reached the specified duration, the working state of the time window can be the open state. After the count value reaches the specified duration, if the time window is not closed, it can enter the overdue state.

[0053] S130, Update the statistical information for the target time window. The statistical information includes the statistical results of at least one window indicator for the target time window.

[0054] Statistical information is the result of statistical analysis based on indicators within a time window. There can be one or more indicators within a window, and the specific selection of these indicators can be set according to business needs. For example, window indicators may include the number of times a user clicks on objects on a page, the number of times an object is searched, the number of sessions, or other indicators. The objects are related to the functions of the application or platform. For example, in a real estate transaction application, objects include new home listings, used home listings, and rental listings.

[0055] Whenever a new stream data entry is added to the target time window, the statistics for that time window are updated, resulting in the latest statistical results for the window metrics. These results can be cumulative. For example, if stream data k3 is added to the target time window, bringing the total number of stream data entries to k1 through k3, then the values ​​for object clicks, object searches, session counts, or other metrics can be incremented by 1 or n to update the window metric values ​​and obtain the latest statistical results.

[0056] S140, if the update of the statistical information causes the event triggering condition to be met, then the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

[0057] Each update to the statistics for the target time window triggers a check to determine whether the event triggering conditions are met. For example, if the window metric only includes the number of sessions, then whenever a user sends a session message to the agent, the business system receives a stream of data. If the target time window has not expired, updating the statistics will increment the session count by 1, and then the session count is used to determine whether the event triggering conditions have been met.

[0058] Event triggering conditions can be set to improve the accuracy of streaming data processing and reduce processing costs. For example, an event triggering condition could be whether the number of sessions (represented by statistics) has reached a certain threshold. If the number of sessions reaches the set threshold, multiple streams of user behavior data within the target time window are input into a language model (e.g., a pre-trained generative model) to generate user labels. This can improve the accuracy of the labels and reduce the cost of calling the model.

[0059] The streaming data processing method proposed according to the embodiments of this disclosure can support delayed data processing. Driven by the event occurrence time of streaming data, the delayed streaming data is correctly assigned to the corresponding time window, so that the delayed streaming data can be correctly counted, ensuring the accuracy of the statistical results and the correctness of the business logic, and avoiding deviations in the processing results of downstream business logic.

[0060] Figure 3 A schematic flowchart of a streaming data processing method M100, based on some embodiments of this disclosure, is shown. (See attached diagram.) Figure 3 In step S120, the specific steps for determining the target time window to which the streaming data belongs by using the user ID and timestamp may include step S121.

[0061] S121, if no stream data belonging to the user ID has been received before the stream data is received, then open the time window corresponding to the user ID and use it as the target time window to which the stream data belongs.

[0062] Assuming user u1 has never used a certain business application before, when user u1 opens the application for the first time and sends a message to the agent for the first time during use, the resulting stream data k1 is sent to the business system, and the metadata y1 of stream data k1 is obtained. The business system then determines whether user u1's ID is included in the existing user IDs. If user u1's ID is not included in the existing user IDs, it means that no stream data belonging to user u1 has been received before receiving stream data k1. At this time, the system can open a first time window s1 dedicated to user u1, and then allocate the stream data k1 belonging to user u1 to this dedicated time window s1. This dedicated time window s1 is the current target time window s1, and the lifecycle start time (window start time) of the target time window s1 can be the timestamp of stream data k1. Step S121 provides a method for opening time windows and determining target time windows for new users.

[0063] Continue reading Figure 3 In step S121, after opening the time window corresponding to the user ID and using it as the target time window for the streaming data, the timestamp can also be used as the base time for the target time window. The base time is used to determine whether the time window is closed.

[0064] The base time is an attribute or field of a time window. Its value determines whether the time window is currently closed or open. A closed time window will not include new streaming data in its statistics. For the same user, whenever a new time window is opened, its base time can be determined to characterize the time window's stage in its lifecycle.

[0065] Before the statistics of time window s1 are updated for the first time, the window metrics of time window s1 can be initial values ​​or default values. For example, for frequency-related window metrics such as session count and click count, the default value is 0. Each time an update is completed, the values ​​of these window metrics are increased by 1 or by other values. By setting the timestamp of the first stream data in the time window as the base time of the time window, a way to identify the current working status of the time window and whether its life cycle has ended is provided, and dynamic windowing is implemented.

[0066] In step S130, before the statistical information of the newly opened target time window is updated for the first time, the values ​​of at least some window indicators in the statistical information can be default values.

[0067] Since this is the first time user u1's streaming data has been received, time window s1 is also newly opened. Before the initial update of the statistics for time window s1, the values ​​of each window metric are initial values. After the initial update, the values ​​of the window metrics usually increase. For example, for the window metric of session count, the initial value of session count can be 0. After the initial metric statistics are completed, since the streaming data includes the content of one session sent by user u1 to the agent, the session count in the statistics changes from 0 to 1, indicating that one user session has been accumulated in time window s1. Configuring default values ​​provides a unified starting point for the statistics and accumulation of window metrics, ensuring the consistency of the statistical results.

[0068] Continue reading Figure 3 In step S140, if the event triggering condition is met, the base time can be reset to the maximum allowed value. The base time reset to the maximum value indicates that the corresponding time window is in a closed state.

[0069] If the statistics of the target time window s1 do not meet the event triggering conditions, it indicates that the target time window is either normally open or expired, but not closed. In this case, the base time remains unchanged as the timestamp of the first stream data k1 of the target time window s1. After updating the statistics of time window s1, if the event triggering conditions are met, the multiple stream data contained in the target time window s1 need to be sent to downstream logic for processing (e.g., generating user tags). At this time, the target time window s1 can indicate that it is closed by resetting the base time. The method for resetting the base time is to adjust the base time from one timestamp to the maximum value max.

[0070] When new streaming data is received from user u1, the reference time indicates that the currently opened target time window has closed, and a new time window needs to be opened to receive the new streaming data. Therefore, by assigning and resetting the reference time, the working state of the time window is adjusted. This allows for the continuous addition of new time windows to receive new streaming data as it continues to be generated, achieving a closed-loop process for streaming data processing and dynamic window closing.

[0071] It should be noted that in the streaming data processing method provided in this disclosure, the overdue state and the closed state of the time window are not the same. The target time window may not necessarily be closed after it expires, but it can continue to wait for the arrival of the next stream data, and the overdue target time window will only be closed after the next stream data is counted into the overdue target time window.

[0072] Figure 4A schematic flowchart of a streaming data processing method M100, based on some embodiments of this disclosure, is shown in the case of non-first-time reception of streaming data from user u1. (See also...) Figure 4 In step S120, the specific steps for determining the target time window to which the streaming data belongs using the user ID and timestamp may include steps S122 and S123. It should be noted that for the same stream of data received by the system, steps S121 and steps S122+S123 are selectively executed.

[0073] S122, if stream data belonging to user ID has already been received before the current stream data is received, then the timestamp and the reference time of the most recently opened time window corresponding to user ID are compared numerically. The reference time is used to determine whether the time window is closed.

[0074] S123, determine the target time window to which the streaming data belongs based on the comparison results.

[0075] For example, after the system first opens a time window corresponding to user u1, it receives stream data k2 from user u1. Stream data k2 is the stream data formed by user u1 sending a session message to the agent for the second time. The business system will determine whether the existing user IDs contain user u1's ID. If it is determined that the existing user IDs contain user u1's ID, it means that stream data belonging to user u1 has already been received before receiving stream data k2. At this time, the newly opened time window is time window s1, and the base time of time window s1 is the timestamp of stream data k1. By comparing the timestamp of stream data k2 with the base time of time window s1 (the timestamp of stream data k1), it can be determined whether the target time window of stream data k2 is the existing time window s1 or a new time window s2 needs to be opened. The one with the smaller time value was formed earlier, and the one with the larger time value was formed later. This determines the target time window, enabling the determination of the target time window for each stream data.

[0076] Figure 5 A schematic flowchart of a streaming data processing method M100 in some other embodiments of this disclosure is shown, addressing the case where streaming data from user u1 is received for the first time. (See also...) Figure 5In step S123, the method for determining the target time window to which the streaming data belongs based on the comparison result may specifically include: if the timestamp is later than the reference time, and the newly opened time window is not closed as determined by the reference time, then the newly opened time window is taken as the target time window to which the streaming data belongs; if the timestamp is not later than the reference time, then a new time window corresponding to the user ID is opened and taken as the target time window to which the streaming data belongs. After opening the new time window corresponding to the user ID and taking it as the target time window to which the streaming data belongs, the timestamp of the streaming data can be used as the reference time of the new time window.

[0077] Determining the target time window by comparing time values ​​involves several scenarios. In the first scenario, the timestamp of stream data k2 is greater than the base time (timestamp of stream data k1) of time window s1, indicating that stream data k2 was formed later than stream data k1. Furthermore, time window s1 is normally open, meaning user u1 quickly sent a message to the agent again after forming stream data k1, thus forming stream data k2. When the system receives stream data k2, time window s1 has not yet expired, indicating that there is no significant communication delay for stream data k2. In this case, the newly opened time window is time window s1, so time window s1 can be directly used as the target time window for stream data k2. Therefore, time window s1 includes both stream data k1 and stream data k2.

[0078] In the second scenario, the timestamp of streaming data k2 is much larger than the base time of time window s1, and time window s1 has expired. This indicates that user u1 sent a message to the agent again to form streaming data k2 after a long interval following the formation of streaming data k1. In this case, the newly opened time window is time window s1. Although time window s1 has expired, it can still be used as the target time window for streaming data k2.

[0079] In the third scenario, the timestamp of streaming data k2 is only slightly greater than the base time of time window s1, but time window s1 has already expired. This indicates that user u1 quickly sends another message to the agent after forming streaming data k1, thus forming streaming data k2. However, due to communication delays and other reasons, time window s1 has already expired when the system receives streaming data k2. The newly opened time window at this point is time window s1. Although time window s1 has expired, it can still be used as the target time window for streaming data k2.

[0080] In the fourth scenario, the timestamp value of stream data k2 is less than the base time value of time window s1. Since the comparison is based on timestamps, and stream data k2 was created later than stream data k1, the timestamp value of stream data k2 will only be less than the base time value when the base time is the maximum value. This indicates that time window s1 is closed. Therefore, a new time window s2 needs to be opened, which is called dynamic windowing. Time window s2 is used as the target time window for stream data k2, and stream data k2 is also the first stream data received by time window s2.

[0081] Furthermore, in the fourth scenario described above, after the target time window is determined through steps S122 and S123, before the initial update of the statistics for the newly opened target time window (step S130), at least some window metrics in the statistics can have default values. And in all the scenarios described above, after the target time window is determined through steps S122 and S123 and the statistics are updated, if the update of the statistics causes the event triggering condition to be met, then in addition to triggering the processing flow of the target time window s1, the base time of the target time window s1 will also be reset to the maximum allowed value.

[0082] Step S122 provides a variety of methods for determining the target time window under different conditions, which can ensure that the correct time window to which the streaming data belongs can be determined under different operating conditions.

[0083] Figure 6 A schematic flowchart of a streaming data processing method M100 according to other embodiments of this disclosure is shown. (See also...) Figure 6 In step S130, the method for updating the statistical information of the target time window can be as follows: perform incremental aggregation calculations on the window indicators of the target time window through the aggregation engine to complete the update of the statistical information of the target time window.

[0084] An aggregation engine is a software component or system module capable of summarizing, statistically analyzing, calculating, and compressing data. Aggregation engines can be implemented using Apache Paimon technology. Paimon is a stream-batch integrated lake table format, suitable for data processing in real-time data lake scenarios. Statistical information for the target time window can exist in tabular form. Through Paimon's native aggregation capabilities, during the process of writing to the statistical information table (i.e., updating statistical information), the corresponding behavioral events (stream data) are incrementally aggregated in real time using different aggregation functions (such as SUM, COUNT, etc.) with the user ID as the primary key. This generates multi-dimensional summary statistical results, where each dimension corresponds to a field, or a window metric.

[0085] Compared to using key-value stores (such as Redis) to generate and store statistics, using an aggregation engine eliminates the need to package each field into a single key-value pair. It offers stronger support for complex statistical metrics, solves the problem of key-value stores being unable to update a single field, and can implement various aggregation statistical logics at a lower cost without the need to write additional function logic or perform corresponding development and maintenance.

[0086] For example, the window metric may include the number of times the target time window is allocated with the streaming data. Accordingly, the event triggering condition may include: the number of allocations is not less than a threshold.

[0087] Whenever a time window is allocated a stream of data representing a user's session message, the number of streams included in the time window is incremented by 1. Furthermore, when the statistics for that time window are updated, the allocation count in the window metrics is also incremented by 1. When the allocation count reaches a threshold, the event trigger condition is considered met, and processing procedures such as user tag generation are initiated based on the latest statistics for the stream data within the current target time window.

[0088] The event triggering condition may also include: the target time window is in an expired state. In other words, the event triggering condition can include the following two items: the number of allocations is not less than the threshold, and the target time window is in an expired state. The event triggering condition is met if either of these conditions is met. If the target time window's working state becomes expired, regardless of whether the number of allocations (count) for target time window s1 has reached the threshold, the processing flow for the streaming data within target time window s1 will be triggered, and the base time of target time window s1 will be reset to its maximum value, thereby closing target time window s1.

[0089] Figure 7 A schematic flowchart of a streaming data processing method M100 according to other embodiments of this disclosure is shown. (See also...) Figure 7 After updating the statistical information for the target time window (step S130), the streaming data processing method M100 may further include the following steps: if a change in the base time causes the event triggering condition to be met, then the processing flow for the streaming data belonging to the target time window is triggered based on the latest statistical information for the target time window. In other words, Step S140 can be specifically described as follows: If the update of statistical information or the change in the working status of the target time window causes the event triggering condition to be met, then the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

[0090] After the statistics are updated (step S130), if the number of allocations reaches the threshold, step S140 will determine that the event triggering condition is met, and the subsequent streaming data processing flow will begin. After determining the target time window (step S120), if it is found that the working status of the target time window s1 has expired, after completing the update of the statistics (step S130), step S140 will also determine that the event triggering condition is met, and the subsequent streaming data processing flow will begin. Since the event triggering condition is met through expiration rather than through allocation, the amount of streaming data belonging to the target time window is less than the amount of streaming data in the target time window when the event triggering condition is met through allocation. This less-than-normal amount of streaming data is processed, and real-time user tags are extracted using a language model or other artificial intelligence models.

[0091] It is understandable that whether the event triggering conditions are met due to the update of statistical information or the change in the working status of the target time window, the base time will become the maximum value, thereby closing the target time window.

[0092] Figure 8 A schematic flowchart of a streaming data processing method M100 according to other embodiments of this disclosure is shown. (See also...) Figure 8 In step S140, the method of triggering the processing flow of streaming data belonging to the target time window based on the latest statistical information of the target time window can specifically include the following steps: generating trigger event information containing a trigger identifier and the latest statistical information of the target time window, wherein the trigger identifier is used to indicate whether the event triggering condition is met; and in response to the generation of the trigger event information, triggering the processing flow of streaming data belonging to the target time window.

[0093] Trigger event information can be stored in a table, and the trigger identifier can be a field in the trigger event table, i.e., the trigger field. Trigger event information can be obtained by copying statistical information and changing the value of the trigger field. After the trigger event information is generated and stored in the corresponding data table, the data in the table, along with the corresponding streaming data, can be sent to the downstream language model for user tag generation; alternatively, the downstream system can monitor the data table storing trigger event information in real time, and when the data table undergoes incremental updates, the downstream system actively retrieves the newly generated trigger event information from the data table.

[0094] Different event triggering conditions can be represented by different trigger fields. For example, when an event triggering condition includes both "the number of allocations is not less than the threshold" and "the target time window is in an overdue state," there can be two trigger fields, each corresponding to one of the two event triggering conditions. The values ​​of the trigger fields differ depending on whether the event triggering condition is not met or has been met, thus allowing the trigger field to indicate whether the event triggering condition has been met.

[0095] For example, at least one of the streaming data, statistics, and trigger event information can be stored in a storage space other than memory in the form of structured data. The structured data can be a data table in a relational database, or a spreadsheet or a JSON (JavaScript Object Notation) file. Streaming data containing user session content, statistics containing window metric values, and event trigger information containing window metric values ​​and trigger field values ​​can all be stored in the business server or an external database in the form of database tables.

[0096] A data table storing streaming data can be called a data detail table. The data detail table can include fields such as session record ID (record_id), user ID (ucid), streaming data address (url), and timestamp (event_time). The session record ID field can be the primary key of the data detail table. The data detail table can be obtained after ETL (Extract, Transform, Load) cleaning of the raw streaming data, and can store the full dataset. ETL cleaning is used to extract raw data, perform cleaning and formatting operations on the raw data, and load the processed data into the target system.

[0097] A data table that stores statistical information can be called a statistics table. A statistics table can include fields such as user ID (ucid), window start time (window_time), and number of allocations (count). The user ID field can be the primary key of the details table.

[0098] A data table storing event trigger information can be called an event trigger table. The event trigger table can include fields such as User ID (ucid), Window Start Time (window_time), Number of Allocations (count), as well as a first trigger field and a second trigger field. The first trigger field indicates whether the number of allocations is not less than a threshold, and the second trigger field indicates whether the target time window has expired. The User ID field and the Window Start Time field can be the primary key of the Window Start Time.

[0099] Alternatively, the base time can also be stored in a data table, called a maintenance table. The maintenance table can include fields for user ID (ucid) and base time. The user ID can be the primary key.

[0100] It is understood that the table structure fields of the aforementioned data detail table, data statistics table, and event trigger table may be simplified to minimize the requirements for implementing the disclosed solution.

[0101] This implementation can represent the state of a time window in the form of a structured data table. Each data table has a clear primary key and multiple columns of attributes. Each statistical dimension (such as frequency, duration, average, etc.) in the data statistics table is an independent field. Window indicators can exist in a decomposable, composable, and computable relational data manner, and natively support updating and inserting single fields of the data table. It can be deeply integrated with aggregation engines (such as Paimon) to support native aggregation, without requiring developers to manually write calculation logic at the application layer.

[0102] Storing this information in a database, compared to Redis's approach of storing all data in memory, eliminates the need for memory caching of the entire dataset, making it suitable for large-scale data scenarios. Adding new statistical dimensions, such as window metrics, does not increase memory usage, reducing memory requirements and supporting users with large datasets. Furthermore, it offers good support for long-term business cycles; for monthly and quarterly statistics, user streaming data is stored in the database rather than being cached in memory long-term, minimizing memory consumption. In addition, this implementation method can also be applied to the Flink framework, which is an open-source stream processing framework that supports high throughput, low latency, and exactly-once semantics for data processing, and can provide a real-time data processing engine.

[0103] For example, a business system can be configured with specified parameters. Before executing certain steps, it must first determine whether the value of the specified parameter corresponds to the value of that step. The step will only be executed if the specified parameter's value is the value corresponding to that step; otherwise, it will not be executed. The specified parameter is used to represent the data flow order. By restricting the execution order of steps through the value of the specified parameter, the specified step can only be executed after the preceding step has been completed. This avoids cyclical triggering between different steps and prevents the dynamic window opening and closing process from getting stuck in an infinite loop. This solves the problem of not supporting window back and rollback in the case of dynamic window opening in a streaming scenario.

[0104] When updating the statistics of a newly opened target time window for the first time, the value of the specified parameter can be updated to the first value only if the value of the specified parameter is the initial value.

[0105] It can trigger actions such as using the timestamp as the base time for the new time window and determining whether the event triggering conditions are met, only when the value of the specified parameter is the first value.

[0106] When generating the event information, the value of the specified parameter is updated to the second value. It can trigger the action of resetting the statistical results in the statistics of the target time window and the action of updating the value of the specified parameter to the initial value only when the value of the specified parameter is the second value.

[0107] It can trigger the action of resetting the base time only when the value of the specified parameter is the initial value.

[0108] For example, after the system receives new streaming data k1, opens a new time window s1, and sets the new time window s1 as the target time window to which the streaming data belongs, since the target time window only contains streaming data k1, updating the statistics (executing step S130) is the first update of the statistics for time window s1. Therefore, before executing step S130, it is necessary to determine whether the specified parameter is the initial value. If it is, step S130 is executed normally; otherwise, step S130 is not executed.

[0109] After completing step S130, the specified parameter is changed from its initial value to its first value. Since the base time is currently at its maximum value, changing the specified parameter to its first value allows the base time of the target time window to be changed from the maximum value to the timestamp of the streaming data k1. At this point, it can also be determined whether the event triggering condition is met. Because the number of allocations has not reached the threshold of 3, the event triggering condition is not yet met.

[0110] In the subsequent process, the system receives new stream data k2 and updates the statistics, but the number of allocations still does not reach the threshold of 3. Then, the system receives new stream data k2 again and updates the statistics, and the number of allocations reaches 3, thus confirming that the event triggering event has been met, and generating trigger event information. After generating the trigger event information, the value of the specified parameter is changed from the first value to the second value. Therefore, the system can reset the statistics of time window s1 to their initial value, for example, resetting the number of allocations to 0, and resetting the value of the specified parameter from the second value to the initial value, completing the closed loop of the stream data processing method.

[0111] After generating the trigger event information, user tags are extracted from the stream data k1, k2, and k3 contained in time window s1, and time window s1 is closed. Therefore, when the specified parameter is the initial value, the base time can be reset from the timestamp of stream data k1 to max, indicating that time window s1 is closed. When the system receives a new data stream k4, a new time window s2 is opened and a new cycle begins.

[0112] This disclosure provides an adaptive window mechanism driven by business events, enabled on demand, spanning long periods, and supporting delayed data backfilling. Under the dynamic window mechanism, there may be bidirectional data flow between state updates and result output, which may lead to problems such as cyclic triggering or state inconsistency due to lack of concurrency control. Through the streaming data processing method of this embodiment, dynamic session windows can be accurately managed based on event time in a streaming environment, supporting multi-dimensional efficient aggregation, compatible with delayed data processing, possessing state traceability, and avoiding computational dead loops.

[0113] Figure 9 An overall flowchart of a method for processing streaming data according to some embodiments of the present disclosure is shown. Figure 9 The detailed description provided is for the purpose of better understanding the technical solution of this disclosure and should not be considered as a limitation on the scope of protection of this disclosure. In the process of implementing the technical concept of this disclosure, one or more steps may be omitted, or other alternative methods may be adopted.

[0114] When user u1 engages in a conversation with the application's agent on the terminal device, user u1 enters and sends a message in the dialog box. The business system extracts the message entered by user u1 from the conversation content to obtain raw data, then performs ETL cleaning on the raw data, and generates a data detail table based on the cleaned result. The data detail table is streaming data k7, which includes the session record ID (record_id), user ID (ucid), streaming data address (url), and timestamp (event_time). The timestamp is 10:00.

[0115] After receiving stream data k7, the system queries the maintenance table containing the user ID and base time to confirm that the user ID already exists, indicating that stream data for that user ID has been received previously (corresponding to ① query). At this time, the base time is max, so a new time window s5 is opened, and stream data k7 is allocated to time window s5. The system then writes the user ID, timestamp t1, and specified parameter of stream data k7 to the data statistics table. The value of the specified parameter changes from the initial value of 3 to the first value of 1, and the allocation count is increased from the initial value of 0 to 1 (corresponding to ② write). Since the specified parameter in the data statistics table is the first value of 1, it is possible to determine whether the triggering condition is met. The system determines that the count value has not reached 3, so no triggering occurs. Furthermore, since the specified parameter in the data statistics table is 1, the base time in the maintenance table can be changed from max to timestamp t1 (corresponding to ⑥ update).

[0116] The system then receives streaming data k8. At this time, the base time is t1. The timestamp of streaming data k8 is later than the base time t1, so streaming data k8 is assigned to time window s5, and the data statistics table is updated, making the count value 2. It is determined that the count value has not reached 3, so no triggering is performed.

[0117] Afterwards, the system receives streaming data k9. At this point, the base time is still t1. Since the timestamp of streaming data k9 is later than the base time t1, streaming data k9 is assigned to time window s5, and the data statistics table is updated, making the count value 3. After determining that the count value has reached 3, data is written to the event trigger table (corresponding to ③ writing). The event trigger table includes the user ID (ucid), window start time (window_time), number of allocations (count), first trigger field, second trigger field, and specified parameter. The window start time is the timestamp t1, the number of allocations is 3, the first trigger field is 1 (triggered by the number of allocations), the second trigger field is 0, and the specified parameter is the second value 2. It can be understood that the specified parameter in the data statistics table is still 1 at this point, but the next step will not use the specified parameter in the data statistics table; instead, it will use the specified parameter in the event trigger table.

[0118] Since the specified parameter in the event trigger table is the second value 2, the data statistics table is reset, changing the allocation count from 3 to 0, and the specified parameter in the data statistics table changes from the first value 1 to the initial value 3 (corresponding to ④ reset). Additionally, since the specified parameter in the data statistics table is the initial value 3, the base time in the maintenance table can be reset, changing the base time from t1 to the max value (corresponding to ⑤ reset), and the time window s5 changes from open to closed, thus achieving a closed loop in the streaming data processing process. In the above process, a total of 3 stream data were input into the language model for data processing and user tag extraction.

[0119] Suppose that when the system receives the aforementioned streaming data k8, the time window s5 has expired due to communication delay. At this point, the base time is t1, and the timestamp of streaming data k8 is later than the base time t1. Therefore, streaming data k8 is assigned to time window s5, and the data statistics table is updated, making the count value 2. Since the specified parameter in the data statistics table is the first value 1, it is possible to determine whether the triggering condition is met. It is determined that the count value has not reached 3, but that time window s5 is already expired. Therefore, data is written to the event trigger table (corresponding to ③ writing). The first trigger field in the event trigger table is 0, the second trigger field is 1 (triggered by expiration), and the specified parameter is the second value 2.

[0120] Since the specified parameter in the event trigger table is the second value 2, the data statistics table is reset, changing the allocation count from 1 to 0, and the specified parameter in the data statistics table changes from the first value 1 to the initial value 3 (corresponding to ④ reset). Additionally, since the specified parameter in the data statistics table is the initial value 3, the base time in the maintenance table can be reset, changing the base time from t1 to the max value (corresponding to ⑤ reset), and the time window s5 changes from expired to closed, thus achieving a closed loop in the streaming data processing process. In the above process, only two stream data entries were input into the language model for data processing and user tag extraction.

[0121] During the user tag extraction process, the system obtains historical session content from the above 3 or 2 streams of data, including messages input by the user to the agent, and then extracts user tags through a language model to update the user profile.

[0122] Based on any of the above embodiments, this disclosure also provides a streaming data processing apparatus. Figure 10 This is a schematic block diagram of a streaming data processing apparatus according to one embodiment of this disclosure. Figure 10 As shown, the data processing device includes: a metadata determination module 110, a time window determination module 120, a statistical information update module 130, and a processing flow triggering module 140.

[0123] The metadata determination module 110 is used to determine the metadata of the stream data in response to receiving the user's stream data, wherein the metadata includes the user ID and the timestamp of the stream data.

[0124] The time window determination module 120 determines the target time window to which the streaming data belongs by using the user ID and timestamp.

[0125] The statistical information update module 130 is used to update the statistical information of the target time window, wherein the statistical information includes the statistical results of at least one window indicator of the target time window.

[0126] If the update of statistical information causes the event triggering condition to be met, the processing flow triggering module 140 will trigger the processing flow of the streaming data belonging to the target time window based on the latest statistical information of the target time window.

[0127] The aforementioned streaming data processing device can be in the form of computer software, and each module of the device can be implemented through computer software modules. The specific implementation process of the functions and roles of each module in the aforementioned device is detailed in the corresponding steps of the above method, and will not be repeated here.

[0128] The entity executing the streaming data processing method in the specific embodiments of this disclosure can be an electronic device such as a computer or server. Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the streaming data processing method of any of the embodiments described above.

[0129] Figure 11 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure.

[0130] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0131] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0132] The processor 1200 can be a central processing unit (CPU). The processor 1200 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0133] The memory 1300 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions of a computer program in the embodiments of this disclosure. The processor 1200 implements a streaming data processing method by running the non-transitory software programs, instructions, and modules stored in the memory 1300.

[0134] The memory 1300 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created by the processor 1200, such as streaming data, streaming data metadata, statistics, event triggering information, etc. Furthermore, the memory 1300 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1300 may optionally include memory remotely located relative to the processor 1200, and these remote memories may be connected to the processor 1200 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0135] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0136] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the processes or functions of this disclosure are performed wholly or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, network equipment, user equipment, core network equipment, OAM, or other programmable device.

[0137] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0138] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0143] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0144] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

[0145] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0146] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0147] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0148] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0149] At the same time, it is understood that the data involved in this disclosed technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

Claims

1. A method for processing streaming data, characterized in that, include: In response to receiving user streaming data, determine the metadata of the streaming data, the metadata including the user ID and the timestamp of the streaming data; The target time window to which the streaming data belongs is determined by the user ID and the timestamp; Update the statistical information of the target time window, the statistical information including the statistical results of at least one window indicator of the target time window; as well as If the update of the statistical information causes the event triggering condition to be met, then the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

2. The method for processing streaming data according to claim 1, characterized in that, Determining the target time window to which the streaming data belongs using the user ID and the timestamp includes: If streaming data belonging to the user ID has been received before the streaming data is received, then the timestamp is compared with the reference time of the most recently opened time window corresponding to the user ID, where the reference time is used to determine whether the time window is closed; and The target time window to which the streaming data belongs is determined based on the comparison results.

3. The method for processing streaming data according to claim 2, characterized in that, Determining the target time window to which the streaming data belongs based on the comparison results includes: If the timestamp is later than the reference time, and the reference time determines that the newly opened time window is not closed, then the newly opened time window is taken as the target time window to which the streaming data belongs.

4. The method for processing streaming data according to claim 2, characterized in that, Determining the target time window to which the streaming data belongs based on the comparison results includes: If the timestamp is not later than the base time, a new time window corresponding to the user ID is opened and used as the target time window to which the streaming data belongs.

5. The method for processing streaming data according to any one of claims 2-4, characterized in that, If the event triggering condition is met, the base time will be reset to the maximum allowed value. The base time being reset to the maximum value indicates that the corresponding time window is in a closed state.

6. The method for processing streaming data according to claim 1, characterized in that, The window metric includes: the number of times the target time window is allocated to the stream data, and the event triggering condition includes: the number of allocations is not less than the number threshold.

7. The method for processing streaming data according to any one of claims 1-3, characterized in that, The event triggering conditions include: the target time window is in an overdue state; after updating the statistical information of the target time window, the method further includes: If the working status of the target time window changes such that the event triggering condition is met, then the processing flow of the streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window.

8. The method for processing streaming data according to claim 1, characterized in that, The processing flow for streaming data belonging to the target time window is triggered based on the latest statistical information of the target time window, including: Generate trigger event information containing a trigger identifier and the latest statistical information for the target time window, wherein the trigger identifier indicates whether the event triggering condition has been met; and In response to the generation of the trigger event information, a processing flow for the streaming data belonging to the target time window is triggered.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 8.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, is used to implement the method of any one of claims 1 to 8.

Images