Flow determination method and device, computer equipment, medium and product
By determining the degree of traffic sparseness in the sliding window and selecting an appropriate calculation strategy, the problem of large amount of traffic statistics in the sliding window is solved, and the effect of reducing calculation costs and improving efficiency is achieved.
Patent Information
- Application Number
- CN202310528986.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, the sliding window traffic statistics method has a large amount of calculation, which leads to high overhead of computing resources and makes it difficult to effectively reduce the calculation amount of sliding window traffic information.
By obtaining the task to be detected, the traffic sparseness of the object to be detected in the second sliding window is determined based on the current time, the end time of the first sliding window and the size of the sliding window, and the target traffic calculation strategy is selected based on the sparseness to reduce the calculation amount .
It effectively reduces the amount of traffic information in the sliding window, reduces the overall calculation cost, and improves the calculation efficiency.
Smart Images

Figure CN118945074A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to a flow determination method, a flow determination device, a computer device, a computer-readable storage medium, and a computer program product. Background Art
[0002] With the rapid development of Internet technology, people are increasingly fond of posting or viewing various forms of attention-grabbing events on the Internet (such as video, graphic, etc.). How to conduct traffic statistics on these attention-grabbing events is a major trend nowadays.
[0003] At present, traffic statistics are mostly calculated through sliding windows, which can be mainly divided into two categories. One is event-driven window traffic statistics. The main idea is to perceive upstream events in real time through the window computing service, perform window calculations, and then output the traffic information of the sliding window on time through the timer. This type of method will generate a large number of background threads, and the overall amount of calculation is very large. The other is mini-batch-based traffic statistics. The main idea is to first calculate the traffic within each step, and then traverse it to obtain the traffic information of the sliding window. This type of method needs to traverse the state data in the window each time. When calculating the traffic indicators of a large window, the overall amount of calculation is very large, which brings a large amount of computing resource overhead.
[0004] Therefore, how to effectively reduce the computational complexity of sliding window traffic information and thus effectively save the overall computing cost has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The embodiments of the present application provide a flow determination method, apparatus, computer equipment, medium, and product, which can effectively reduce the amount of calculation of flow information in a sliding window, thereby effectively reducing the calculation cost.
[0006] On the one hand, an embodiment of the present application discloses a flow determination method, the method comprising:
[0007] Acquire a task to be detected, wherein the task to be detected includes an object to be detected and an indicator to be detected;
[0008] If the indicator to be detected of the object to be detected has traffic at the current time, determine the traffic sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, the end time of the first sliding window, and the size of the sliding window, where the first sliding window is a previous sliding window of the second sliding window;
[0009] Determining a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window;
[0010] The flow information of the indicator to be detected of the object to be detected within the second sliding window is determined according to the target flow calculation strategy.
[0011] In one aspect, an embodiment of the present application discloses a flow rate determination device, the device comprising:
[0012] An acquisition unit, used for acquiring a task to be detected, wherein the task to be detected includes an object to be detected and an indicator to be detected;
[0013] a determining unit, configured to determine, if the indicator to be detected of the object to be detected has flow at the current time, a flow sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, an end time of the first sliding window, and a size of the sliding window, wherein the first sliding window is a previous sliding window of the second sliding window;
[0014] The determining unit is further configured to determine a target flow calculation strategy according to a flow sparseness of the indicator to be detected in the second sliding window;
[0015] The determining unit is further configured to determine flow information of the indicator to be detected of the object to be detected within the second sliding window according to the target flow calculation strategy.
[0016] On the one hand, an embodiment of the present application discloses a computer device, which includes a processor suitable for implementing one or more computer programs; and a computer storage medium, wherein the computer storage medium stores one or more computer programs, and the one or more computer programs are suitable for being loaded by the processor and executed in accordance with the above-mentioned flow determination method.
[0017] In one aspect, the present application discloses a computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by a processor and executing the above-mentioned flow determination method.
[0018] In one aspect, the present application discloses a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned flow determination method.
[0019] In the embodiment of the present application, the task to be detected is obtained, and the task to be detected includes an object to be detected and an index to be detected. Through the object to be detected and the index to be detected, the task requirements can be clarified, that is, what results are output. If the index to be detected of the object to be detected has flow at the current time, then according to the current time, the end time of the first sliding window and the sliding window size, the flow sparsity of the index to be detected in the second sliding window corresponding to the object to be detected at the current time is determined, and the first sliding window is the previous sliding window of the second sliding window. This is equivalent to a condition for triggering window calculation. After the previous sliding window completes the flow calculation, when the flow is detected again, the window calculation is triggered, and the flow sparsity of the index to be detected in the second sliding window must first be determined. According to the flow sparsity of the index to be detected in the second sliding window, the target flow calculation strategy is determined, and finally the flow information of the index to be detected in the second sliding window is determined according to the target flow calculation strategy. In this process, different target flow calculation strategies are selected to perform flow calculation according to the sparsity of the flow of the index to be detected in the second sliding window, which can reduce the amount of calculation and effectively reduce the calculation cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0021] Figure 1 It is a schematic diagram of the network architecture of a flow determination system disclosed in an embodiment of the present application;
[0022] Figure 2 It is a flow chart of a flow determination method disclosed in an embodiment of the present application;
[0023] Figure 3 is a schematic diagram of a sliding window disclosed in an embodiment of the present application;
[0024] Figure 4 It is a flow chart of another flow determination method disclosed in an embodiment of the present application;
[0025] Figure 5 is a schematic diagram of a sliding exit and step length disclosed in an embodiment of the present application;
[0026] Figure 6 It is a flow chart of another flow determination method disclosed in an embodiment of the present application;
[0027] Figure 7 It is a structural schematic diagram of a flow determination device disclosed in an embodiment of the present application;
[0028] Figure 8 It is a structural schematic diagram of a computer device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0030] In order to more clearly understand the technical solution provided by the embodiments of the present application, some key terms involved in the embodiments of the present application are first introduced here:
[0031] 1. Sliding Window
[0032] The information in a fixed time (or data) range in the past is processed at intervals. The interval of each slide is called the step length, and the fixed time range is called the size of the sliding window. For example, if the flow rate in the past 24 hours is calculated and updated every minute, the step length is 1 minute and the sliding window size is 24 hours.
[0033] 2. Window key
[0034] It is an attribute or attribute group in the original data. It is a dimension that the business is very concerned about. The sliding window computing service will calculate the indicators of each key in the past period based on the key. For example, when the business needs to calculate the 24-hour sliding window indicators of the content every minute, the content ID is the window key. More simply, it can be understood as the identifier of a video or a news in the original data.
[0035] 3. Status
[0036] When a real-time streaming program performs real-time calculations, if it relies on previous information, the intermediate information cached in memory is called state. For example, when calculating a sliding window, the previous traffic information is cached.
[0037] 4. Cloud Technology
[0038] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. Cloud storage is a new concept extended and developed from the concept of cloud computing. Distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide data storage and business access functions to the outside world. In short, a database can be regarded as an electronic filing cabinet - a place to store electronic files. Users can add, query, update, delete and other operations on the data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
[0039] 5. Blockchain
[0040] Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of the information (anti-counterfeiting) and generate the next block.
[0041] The embodiment of the present application provides a flow determination scheme, which includes: ① obtaining a task to be detected, the task to be detected includes an object to be detected and an indicator to be detected, and through the object to be detected and the indicator to be detected, the task requirements can be clarified, that is, what results to output. ② If there is flow in the indicator to be detected of the object to be detected at the current time, then according to the current time, the end time of the first sliding window and the size of the sliding window, the flow sparseness of the indicator to be detected in the second sliding window corresponding to the current time of the object to be detected is determined, and the first sliding window is the previous sliding window of the second sliding window. This is equivalent to a condition for triggering window calculation. After the flow calculation of the previous sliding window is completed, if the flow is detected again after a certain number of steps, the window calculation is triggered. First, the flow sparseness of the indicator to be detected in the second sliding window must be determined. ③ Determine the target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window. ④ Determine the flow information of the indicator to be detected in the second sliding window according to the target flow calculation strategy. In this process, different target flow calculation strategies are selected for flow calculation according to the sparseness of the flow of the indicator to be detected in the second sliding window, which can reduce the amount of calculation and effectively reduce the calculation cost.
[0042] In a possible implementation, the flow determination method provided in the present application can be implemented based on cloud technology and / or blockchain technology. Specifically, it may involve one or more of cloud storage, cloud database, and big data in cloud technology. For example, the data required to execute the flow determination method (such as the task to be detected and the end time of the first sliding window, etc.) is obtained from the cloud database.
[0043] In a possible implementation, the flow determination method provided by the present application can also be implemented based on blockchain technology. For example, the data required to execute the flow determination method can be stored in the form of blocks on the blockchain; the data generated by executing the flow determination method (such as the flow information of the indicator to be detected of the object to be detected in the second sliding window) can be stored in the form of blocks on the blockchain; in addition, the flow determination device that executes the flow determination method can be a node device in the blockchain network.
[0044] See also Figure 1, is a schematic diagram of the architecture of a flow determination system disclosed in an embodiment of the present application, and the flow determination system may include a first terminal device 101, a second terminal device 102, and a server 103. Different users corresponding to the first terminal device 101 and the second terminal device 102, users can publish content on the first terminal device 101 and the second terminal device 102, and can also watch content on the first terminal device 101 and the second terminal device 102. The server 103 is mainly used to obtain a task to be detected including an object to be detected and an indicator to be detected, and then when it is detected that the indicator to be detected of the object to be detected has flow at the current time, determine the flow sparseness of the indicator to be detected in the sliding window corresponding to the current time, and then use different target flow calculation strategies according to different flow sparseness to determine the flow information of the indicator to be detected in the sliding window corresponding to the current time. In one embodiment, the first terminal device 101 and the second terminal device 102 include but are not limited to: smart phones, tablet computers, smart wearable devices, smart voice interaction devices, smart home appliances, personal computers, vehicle-mounted terminals, smart cameras, virtual reality devices (such as AR (Augmented Reality, Augmented Reality) devices) and other devices, and this application does not limit this. The server 103 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, which are not limited in the embodiments of the present application. The first terminal device 101 and the second terminal device 102 may establish a direct communication connection with the server 103 by wired communication, or may establish an indirect communication connection by wireless communication, which are not limited in the embodiments of the present application.
[0045] In an embodiment of the present application, in combination with the traffic determination system, the traffic determination method may include: the server 103 obtains the task to be detected from the first terminal device 101, the task to be detected includes the object to be detected and the indicator to be detected, and the task requirements, that is, what result to output, can be clarified through the object to be detected and the indicator to be detected; if there is traffic for the indicator to be detected of the object to be detected at the current time, the server 103 determines the traffic sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time according to the current time, the end time of the first sliding window and the size of the sliding window, the first sliding window is the previous sliding window of the second sliding window, which is equivalent to a condition for triggering window calculation. After the traffic calculation of the previous sliding window is completed, when the traffic is detected again, the window calculation is triggered. First, the traffic sparseness of the indicator to be detected in the second sliding window is determined; then the target traffic calculation strategy is determined according to the traffic sparseness of the indicator to be detected in the second sliding window; finally, the traffic information of the indicator to be detected of the object to be detected in the second sliding window is determined according to the target traffic calculation strategy. In this process, different target traffic calculation strategies are selected to perform traffic calculation according to the sparsity of the indicator traffic to be detected in the second sliding window, which can reduce the amount of calculation and effectively reduce the calculation cost.
[0046] In one implementation, the traffic determination method can be applied to various application scenarios, including content distribution scenarios, process control scenarios, and content management scenarios. The following briefly describes each scenario in conjunction with the traffic determination system:
[0047] Content distribution scenario: A task to be detected is obtained from the first terminal device 101, where the task to be detected includes an object to be detected, wherein the object to be detected includes multiple videos and multiple articles. Then the server 103 uses the traffic determination method provided in the embodiment of the present application to determine the traffic information of each video and each article in turn (such as the number of clicks, the number of comments, and the number of forwardings, etc.), and then sorts them, and sends the videos and articles with higher popularity among the multiple videos and multiple articles to the second terminal device 102.
[0048] Process control scenario: Obtain the task to be detected from the first terminal device 101. The object to be detected included in the task to be detected may refer to an article or video of a specified type. Then the server 103 uses the flow determination method provided in the embodiment of the present application to determine the flow information (such as click volume, comment volume, and forwarding volume, etc.) of the specified type of article or video, and then manages its process based on the flow information. If it is for some videos or articles with a relatively bad impact, when its flow information is too large, such videos or articles can be directly offline. Similarly, if it is for some positive videos or articles, they can be promoted.
[0049] Content management scenario: A task to be detected is obtained from the first terminal device 101, where the task to be detected includes an object to be detected, wherein the object to be detected includes multiple videos and multiple articles. Then the server 103 uses the traffic determination method provided in the embodiment of the present application to determine the traffic information of each video and each article in turn (such as the number of clicks, the number of comments, and the number of forwardings, etc.), and grasp the popularity of each video and each article in real time, so that when the popularity of a certain type of video or article is about to bring a bad influence, it can be managed in time to avoid causing adverse effects.
[0050] In a feasible implementation, the flow determination method provided in the embodiment of the present application can be integrated into the streaming service component as a general solution. Most businesses will develop related services based on the streaming service component. The flow determination method provided in the embodiment of the present application can be integrated into the streaming service component. The flow determination device of the present application is integrated in the software development kit (Software Development Kit, SDK). The business only needs to call the corresponding capabilities to achieve efficient window calculation. Because the sliding window calculation requirements are relatively large, this application can become one of the key basic capabilities of the streaming service component.
[0051] It should be noted that in the present application, the relevant data involved in the flow determination process, such as the task to be detected, the step characteristic data stored in the database, and the window characteristic data, etc. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent must be obtained, and the relevant data collection, use and processing processes must comply with relevant laws, regulations and standards of the country and region, comply with the principles of legality, legitimacy and necessity, and do not involve the acquisition of data types prohibited or restricted by laws and regulations. In some optional embodiments, the relevant data involved in the embodiments of the present application are obtained after the object is authorized separately. In addition, when obtaining the separate authorization of the object, the purpose of the relevant data involved is indicated to the object.
[0052] The flow determination method proposed in the embodiment of the present application will be introduced in detail below.
[0053] See also Figure 2 , is a flow chart of a flow determination method disclosed in an embodiment of the present application. The flow determination method can be executed by a computer device, and the computer device can be Figure 1 For the server 103 shown, the flow determination method may specifically include but is not limited to the following steps:
[0054] S201: Acquire a task to be detected, where the task to be detected includes an object to be detected and an indicator to be detected.
[0055] Among them, the task to be detected includes an object to be detected and an indicator to be detected. The object to be detected is the window key described above, such as a specific video or a news. In an embodiment of the present application, the number of tasks to be detected may include multiple, that is, the object to be detected may include multiple. The indicator to be detected refers to an indicator that measures the popularity of the object to be detected. Correspondingly, the indicator to be detected may also include multiple, which may include but not limited to comment indicators, playback indicators, forwarding indicators, and collection indicators. Optionally, the task to be detected may also include time indication information of the object to be detected, which is used to indicate the release time of the object to be detected. In summary, the role of the task to be detected is to indicate the object that the server needs to detect and the indicator that needs to be output, that is, to clearly output the statistical information of which indicator of which object.
[0056] In one possible implementation, the task to be detected may be received by the computer device from the terminal device, may be sent to the computer device immediately after a user publishes a content, or may be sent to the computer device after a certain period of time. In another possible implementation, the task to be detected may be automatically obtained by the computer device, or may be triggered to be executed by a timer, and the computer device automatically obtains the task to be detected from the terminal device once every certain period of time.
[0057] S202: If there is traffic for the indicator to be detected of the object to be detected at the current time, determine the traffic sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time according to the current time, the end time of the first sliding window and the size of the sliding window, where the first sliding window is the previous sliding window of the second sliding window.
[0058] In order to effectively reduce the amount of calculation and reduce the calculation cost, the embodiment of the present application proposes an adaptive flow determination method, that is, according to the flow sparsity of the object to be detected in the current sliding window, the optimal flow calculation strategy is selected for flow calculation, which can reduce the amount of calculation of the sliding window and ultimately effectively reduce the cost of sliding window calculation.
[0059] In the specific implementation process, determining the flow sparseness of the indicator to be detected corresponding to the object to be detected in the sliding window corresponding to the current time may include: ① If the indicator to be detected of the object to be detected has flow at the current time, the current time refers to a certain time after the flow calculation is completed in a sliding window, then according to the current time and the end time of the first sliding window, determine the empty flow time interval, the empty flow time interval is a time interval composed of the current time and the end time of the first sliding window, and there is no flow information of the indicator to be detected in this empty flow time interval. Optionally, the empty flow time interval can be 0, that is, after the flow calculation of the previous sliding window is completed, there is also flow in the next step. After the flow calculation is completed in a sliding window, whether there is flow is the condition for triggering the window calculation. If there is flow, the window calculation is performed. ② Determine the sparse judgment threshold according to the sliding window size and the set rules. For the same object to be detected, the sliding window size is fixed, such as 24 hours or 48 hours; for different objects to be detected, the sliding window size can be different. The set rules refer to some conventional calculation rules, including addition, subtraction, multiplication and division. For example, the sliding window size is W, and the setting rule can be divided by 2, then the sparse judgment threshold is W / 2. ③ Determine the traffic sparseness of the indicator to be detected in the second sliding window corresponding to the current time of the object to be detected according to the empty traffic time interval and the sparse judgment threshold. Specifically, if the length of the empty traffic time interval is greater than the sparse judgment threshold, it is determined that the traffic sparseness of the indicator to be detected in the second sliding window of the object to be detected is sparse; if the length of the empty traffic time interval is less than or equal to the sparse judgment threshold, it is determined that the traffic sparseness of the indicator to be detected in the second sliding window of the object to be detected is dense. Among them, the second sliding window is the sliding window corresponding to the current time, and is the sliding window after the first sliding window.
[0060] For example Figure 3 As shown, it is a schematic diagram of a sliding window for a single object to be detected, wherein ts represents the current time (that is, the end time of the second sliding window), W represents the size of the sliding window, ts-W represents the start time of the second sliding window, lastTs represents the end time of the first sliding window, and lastTs-W represents the start time of the first sliding window. The time interval A from lastTs-W to ts-W is the extra time interval of the first sliding window compared to the second sliding window; the time interval B from ts-W to lastTs is the common time interval of the first sliding window and the second sliding window; the time interval C from lastTs to ts is a time interval without traffic information, that is, the empty traffic time interval described above. The traffic sparseness of the second sliding window can be measured by the size of time interval C. The larger the time interval C, the sparser the traffic sparseness, and the smaller the time interval C, the denser the traffic sparseness.
[0061] S203: Determine a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window.
[0062] In a possible implementation, if the flow sparseness of the indicator to be detected in the second sliding window is sparse, the first flow calculation strategy among the preset multiple flow calculation strategies is determined as the target flow calculation strategy. For any sliding window, the first flow calculation strategy includes: skipping the empty flow time interval and accumulating the flow of the indicator to be detected in the current sliding window.
[0063] In another possible implementation, if the flow sparseness of the indicator to be detected in the second sliding window is dense, the second flow calculation strategy among the preset multiple flow calculation strategies is determined as the target flow calculation strategy. For any sliding window, the second flow calculation strategy includes: integrating the flow of the indicator to be detected in the previous sliding window, the flow of the indicator to be detected corresponding to the current time, and the flow of the indicator to be detected corresponding to the reference time interval; the reference time interval is the time interval composed of the start time of the previous sliding window to the start time of the current sliding window.
[0064] In addition to the first flow calculation strategy and the second flow calculation strategy, the preset multiple flow calculation strategies may also include other flow calculation strategies. For example, when the empty flow time interval is larger than the sliding window size, the third flow calculation strategy is adopted, and the third flow calculation strategy includes: using the flow information at the current time as the flow information of the sliding window at the current time.
[0065] S204: Determine the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy.
[0066] In a possible implementation, the window time information and flow information of each sliding window, and the step time information and flow information of each step in each sliding window may be recorded in a database.
[0067] Specifically, when the target flow calculation strategy is the first flow calculation strategy, determining the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy may include: obtaining the first step length corresponding to the empty flow of the indicator to be detected in the second sliding window from the database. In the embodiment of the present application, the first step length corresponding to the empty flow can be understood as the step length corresponding to the empty flow time interval; obtaining the flow of the indicator to be detected of the second step length from the database, and the second step length is each step length in the second sliding window except the first step length; accumulating the flow of the indicator to be detected of the second step length to obtain the flow information of the indicator to be detected of the object to be detected in the second sliding window. The first flow calculation strategy obtains the first step length corresponding to the empty flow in order to skip the first step length when calculating the flow information, so as to save calculation time and reduce calculation cost.
[0068] When the target flow calculation strategy is the second flow calculation strategy, determining the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy may include: obtaining the first flow of the indicator to be detected in the first sliding window from the database, and obtaining the second flow of the indicator to be detected at the current time; determining the reference time interval according to the start time of the first sliding window and the start time of the second sliding window, and determining the third flow of the indicator to be detected in the reference time interval; integrating the first flow, the second flow and the third flow to obtain the flow information of the indicator to be detected of the object to be detected in the second sliding window. The integration processing here specifically refers to first accumulating the first flow and the second flow, and then subtracting the third flow from the accumulated result to obtain the flow information of the indicator to be detected of the object to be detected in the second sliding window.
[0069] Assuming that the empty flow time interval is I and the sparse determination threshold is W / 2 (W is the sliding window size), the first flow calculation strategy and the second flow strategy can be described using formula (1):
[0070]
[0071] As shown in formula (1), when I> / 2, it means that the traffic sparsity of the indicator to be detected in the second sliding window is sparse. At this time, the first traffic calculation strategy is selected, which is to skip the time interval from the first sliding window to the current time when there is no traffic, and accumulate the traffic in each step corresponding to other intervals. When I≤W / 2, it means that the traffic sparsity of the indicator to be detected in the second sliding window is dense. At this time, the second traffic calculation strategy is selected, which is to remove the head and tail and reuse the traffic information of the first sliding window. First, the traffic information of the first sliding window is added to the traffic information of the current time, and then the traffic corresponding to the extra time interval of the first sliding window compared to the second sliding window is subtracted. In addition to the above two strategies, there is also a more extreme case, that is, I>W. In this case, the strategy for determining the traffic information of the indicator to be detected in the second sliding window is to take the traffic information of the indicator to be detected at the current time as the traffic information of the indicator to be detected in the second sliding window. It should be noted that, taking the step size of 1 minute as an example, the traffic information at 10:00 on XX / XX / XX refers to the traffic information from 09:59:01 on XX / XX / XX to 10:00:00 on XX / XX / XX. In formula (1), ts-W+1 to lastTS and lastTs-W+1 to ts-W have the same meaning as the current time.
[0072] In an embodiment of the present application, a task to be detected is obtained, and the task to be detected includes an object to be detected and an index to be detected. Through the object to be detected and the index to be detected, the task requirements can be clarified, that is, what results are output. If there is flow in the index to be detected of the object to be detected at the current time, then according to the current time, the end time of the first sliding window and the sliding window size, the flow sparsity of the index to be detected in the second sliding window corresponding to the object to be detected at the current time is determined, and the first sliding window is the previous sliding window of the second sliding window. This is equivalent to a condition for triggering window calculation. After the previous sliding window completes the flow calculation, when the flow is detected again, the window calculation is triggered, and the flow sparsity of the index to be detected in the second sliding window must first be determined. According to the flow sparsity of the index to be detected in the second sliding window, the target flow calculation strategy is determined, and finally the flow information of the index to be detected in the second sliding window of the object to be detected is determined according to the target flow calculation strategy. In this process, different target flow calculation strategies are selected to perform flow calculation according to the sparsity of the flow of the index to be detected in the second sliding window, which can reduce the calculation amount of the flow information in the sliding window, thereby effectively reducing the calculation cost.
[0073] See also Figure 4, is a flow chart of another flow determination method disclosed in an embodiment of the present application, the flow determination method describes that in the process of determining the flow information of the indicator to be detected of the object to be detected in the second sliding window, it also includes updating the state in the database, and the state update indicates updating the step feature data and the window feature data, including but not limited to the following steps:
[0074] S401: Acquire a task to be detected, where the task to be detected includes an object to be detected and an indicator to be detected.
[0075] S402: If there is traffic for the indicator to be detected of the object to be detected at the current time, determine the traffic sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time according to the current time, the end time of the first sliding window and the size of the sliding window, where the first sliding window is the previous sliding window of the second sliding window.
[0076] S403: Determine a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window.
[0077] S404: Determine the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy.
[0078] The implementation process of the above S401-S404 can be found in Figure 2 The relevant description of the implementation process of S201-S204 in the illustrated embodiment is not repeated here.
[0079] S405: storing the step time information of each step length and the flow information of the indicator to be detected of the object to be detected within each step length in a database.
[0080] In a possible implementation, after determining the object to be detected and the indicator to be detected, the flow information of the indicator to be detected for each step length can be determined according to the sliding window size and the step length, and the flow information of the indicator to be detected for each step length can be obtained from the database. Then the step time information of each step length is obtained at the same time, and the step time information includes the start time and the end time. If the step length is 1 minute, then the step start time of 10:00 on XX / XX / XX is 10:00:00 on XX / XX / XX, and the end time is 10:00:59 on XX / XX / XX. For subsequent flow calculations, the step time information of each step length and the flow information of the indicator to be detected of the object to be detected within each step length can be stored in the database in the form of a key-value pair. Among them, the key is the step time information, and the value is the flow information of the corresponding step length.
[0081] For example, the flow information for each step can be Figure 5As shown, for the object to be detected, assuming the step size is 1 and the sliding window size is 3, Figure 5 In the time range shown, four sliding windows are divided, and the flow information of each step can be obtained based on each step of each sliding window, such as Figure 5 As shown in the figure, the flow information of each step is d[1]=6, d[2]=4, d[3]=7, d[4]=3, d[5]=1, and d[6]=2. The flow information of each step is stored in the database and can be directly traversed when calculating the flow of the sliding window. The flow information of each sliding window is the sum of the flow information of each step in the sliding window. For example, the flow information of sliding window 1 = d[1]+d[2]+d[3]=6+4+7=17.
[0082] It should be noted that the execution order of step S405 is not limited, and it can be after step S401 or after step S403.
[0083] S406: Store the window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected in the second sliding window into a database.
[0084] After determining the flow information of the indicator to be detected in the second sliding window, the window time information of the second sliding window can also be obtained, and the window time information includes the start time and the end time. Taking the sliding window size of 24 hours as an example, the start time of 10:00 on XX / XX / 06 is 10:00:00 on XX / XX / 05, and the end time is 09:59:59 on XX / XX / 06. Then the window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected in the second sliding window are stored in the database. Optionally, the window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected in the second sliding window can also be stored in the database in the form of a key-value pair, wherein the key is the window time information and the value is the flow information of the corresponding window.
[0085] Step S405 and step S406 belong to the state update step. Storing the step flow information and the window flow information in the database or memory can improve the calculation speed of the flow information in the sliding window and reduce the calculation time and cost.
[0086] S407: Determine the survival time of the target data in the database according to the sliding window size, the target data includes one or both of step characteristic data and window characteristic data, the step characteristic data is the step time information of each step and the flow information of the indicator to be detected of the object to be detected within each step, and the window characteristic data is the window time information of each sliding window and the flow information of the indicator to be detected of the object to be detected within each sliding window.
[0087] S408: If the storage time of the target data in the database reaches the survival time, the target data is deleted from the database.
[0088] In order to prevent the data in the database from expanding continuously, the target data stored in the database may be cleared asynchronously. Step S407 and step S408 describe the process of the asynchronous clearing.
[0089] In a possible implementation, the survival time of the target data in the database is first determined according to the sliding window size. The survival time refers to the maximum time that the target data can be stored in the database. The survival time is associated with the sliding window size. For example, when the sliding window size is W, the survival time can be set to 3 / 2W. Taking W as 24 hours as an example, the survival time can be 3 / 2*24=1 / 16 hour. When the storage time of the target data in the database reaches 1 / 16 hour, the target data is deleted from the database. The target data includes one or both of the step feature data and the window feature data.
[0090] In an optional embodiment, the step characteristic data may refer to the storage time point of the flow information of the indicator to be detected of the detected object within each step, that is, the end time of the step; the window characteristic data may refer to the storage time point of the flow information of the indicator to be detected within each window, that is, the end time of the window.
[0091] The embodiment of the present application describes the determination process of the flow determination method and the state update process of the data in the determination process, and stores the step time information of each step length and the flow information of the to-be-detected index of the object to be detected in each step length in the database; stores the window time information of the second sliding window and the flow information of the to-be-detected index of the object to be detected in the second sliding window in the database; determines the survival time of the step feature data and the window feature data in the database according to the sliding window size, and deletes the step feature data and the window feature data from the database if the storage time in the database reaches the survival time. Therefore, through the embodiment of the present application, not only can the amount of calculation be reduced, the calculation cost can be reduced; but also the speed of flow determination can be improved due to the timely update of data.
[0092] The flow determination method disclosed in the embodiment of the present application is introduced below with an example.
[0093] Set the step size to S (such as 1 minute) and the sliding window size to T (such as 24 hours). At the same time, when expressing time, this application will standardize the calculation in units of S. For example, if the step size is 1 minute, then each time node interval is 1 minute. The purpose is to calculate the flow information of the current sliding window corresponding to ts. Based on this, the flow determination method provided in the embodiment of the present application can be as follows: Figure 6As shown, the following steps are included:
[0094] S601: Acquire the task to be detected. By sensing the event of interest in real time, the task to be detected is acquired, including the object to be detected and the indicator information to be detected.
[0095] S602: Mini batch calculation. Each step scheduling triggers a flow calculation, that is, the flow in this step is calculated for each object to be detected, and the step feature data is stored. The step feature data is represented by time and flow information. It can be stored in the form of a key-value pair, represented by d. For example, the flow information corresponding to the step at time t is d(t).
[0096] S603: Adaptively calculate the sliding window index. For each target index to be detected, adaptive calculation is performed based on the traffic sparseness of the sliding window. First, the time interval I between the current time and the end time of the previous sliding window is calculated, and the size between I and W / 2 is determined, and adaptive calculation is performed:
[0097] (1) If I is greater than W / 2, the traffic information of the current sliding window is determined by directly traversing the traffic in the current sliding window and accumulating it. In order to further reduce the amount of calculation, the accumulation skips the place where there is no traffic from the last time to the current time. In extreme cases, if I is greater than W, the traffic information at time ts can be directly taken as the traffic information of the current sliding window.
[0098] (2) If I is less than or equal to W / 2, the flow information of the current sliding window is determined by removing the head and taking the tail and reusing the flow information of the previous sliding window. First, the flow information of the previous sliding window is added to the flow information of the current time point, and then the flow of the previous sliding window in the extra time interval compared to the current sliding window is subtracted to obtain the flow information of the current sliding window.
[0099] S604: Data storage: The flow information and window time information of the sliding window calculated this time are updated to the state of each object to be detected and cached for use in the next calculation of the flow information of the sliding window.
[0100] S605: Asynchronously clean up the historical cache. For relatively old intermediate calculation results, such as data before the sliding window size, perform asynchronous cleanup. To ensure the data in step S603 is available, the lifetime of each data can be set to 3 / 2W. Step S605 Figure 6 It is not shown in the figure because the asynchronous cleanup process can be performed periodically or in real time as time goes by, and is not limited to after specific steps.
[0101] The flow determination method provided in this embodiment can obtain the following benefits:
[0102] 1) Since the embodiments of the present application can be applied to various real-time scenarios, such as content distribution scenarios, process control scenarios, and content management scenarios, the big data service costs of content distribution scenarios, process control scenarios, and content management scenarios can be saved. Calculating the indicators to be detected in the sliding window is a key process in many business scenarios. By reducing the calculation cost of the indicators to be detected in the sliding window, the big data service costs of related businesses can be effectively reduced.
[0103] 2) Improve the effectiveness of big data services. As the cost of sliding window calculations is greatly reduced, some scenarios that cannot be realized in real time due to cost factors can be realized in real time, which can improve the effectiveness of related services at a limited cost.
[0104] 3) Reduce carbon emissions. In scenarios where users have increasing demands for real-time features, reducing the amount of computation of widely used sliding window features can greatly save the number of servers, because server operation consumes a lot of electricity and other resources. By saving servers, energy consumption can be reduced, thereby reducing carbon emissions and achieving green and sustainable development for the whole society.
[0105] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.
[0106] See also Figure 7 , Figure 7 It is a structural schematic diagram of a flow determination device provided in an embodiment of the present application. The flow determination device can be set in the computer device provided in the embodiment of the present application. Figure 6 The flow determination device shown may be a computer program (including program code) running in a computer device, and the flow determination device may be used to execute Figure 2 , Figure 4 as well as Figure 6 Some or all of the steps in the method embodiment shown. Figure 7 , the flow determination device may include the following units:
[0107] An acquisition unit 701 is used to acquire a task to be detected, where the task to be detected includes an object to be detected and an indicator to be detected;
[0108] A determining unit 702 is configured to determine, if there is flow of the indicator to be detected of the object to be detected at the current time, a flow sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, an end time of the first sliding window, and a size of the sliding window, where the first sliding window is a previous sliding window of the second sliding window;
[0109] The determining unit 702 is further configured to determine a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window;
[0110] The determining unit 702 is further configured to determine flow information of the indicator to be detected of the object to be detected within the second sliding window according to the target flow calculation strategy.
[0111] In a possible implementation, when the determination unit 702 determines the flow sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time according to the current time, the end time of the first sliding window, and the sliding window size, it is specifically used to perform the following steps:
[0112] Determine a time interval of empty traffic according to the current time and the end time of the first sliding window;
[0113] Determine a sparse determination threshold according to the sliding window size and a set rule;
[0114] The flow sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time is determined according to the empty flow time interval and the sparse determination threshold.
[0115] In a possible implementation, when the determination unit 702 determines the target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window, it is specifically configured to perform the following steps:
[0116] If the flow sparseness of the indicator to be detected in the second sliding window is sparse, a first flow calculation strategy among the preset multiple flow calculation strategies is determined as the target flow calculation strategy, and the first flow calculation strategy includes: skipping the empty flow time interval and accumulating the flow of the indicator to be detected in the current sliding window;
[0117] If the traffic sparseness of the indicator to be detected in the second sliding window is dense, the second traffic calculation strategy among the preset multiple traffic calculation strategies is determined as the target traffic calculation strategy, and the second traffic calculation strategy includes: integrating the traffic of the indicator to be detected in the previous sliding window, the traffic of the indicator to be detected corresponding to the current time, and the traffic of the indicator to be detected corresponding to the reference time interval; the reference time interval is a time interval composed of the start time of the previous sliding window to the start time of the current sliding window.
[0118] In a possible implementation, the target flow calculation strategy is the first flow calculation strategy, and when the determination unit 702 determines the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy, it is specifically used to perform the following steps:
[0119] Acquire the first step length corresponding to the empty flow of the indicator to be detected in the second sliding window from the database;
[0120] Acquire the flow of the indicator to be detected with a second step length from the database, where the second step length is each step length in the second sliding window except the first step length;
[0121] The flow rate of the indicator to be detected of the second step length is accumulated to obtain the flow rate information of the indicator to be detected of the object to be detected in the second sliding window.
[0122] In a possible implementation, the target flow calculation strategy is the second flow calculation strategy, and when the determination unit 702 determines the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy, it is specifically used to perform the following steps:
[0123] Acquire from a database a first flow rate of the indicator to be detected in the first sliding window, and acquire a second flow rate of the indicator to be detected at the current time;
[0124] Determine a reference time interval according to the start time of the first sliding window and the start time of the second sliding window, and determine a third flow of the indicator to be detected in the reference time interval;
[0125] The first flow, the second flow and the third flow are integrated and processed to obtain flow information of the indicator to be detected of the object to be detected in the second sliding window.
[0126] In a possible implementation, the determining unit 702 is further configured to determine flow information of the indicator to be detected of each step length of the object to be detected in the second sliding window;
[0127] The acquiring unit 701 is further configured to acquire step time information of each step length in the second sliding window, where the step time information includes a start time and an end time;
[0128] The processing unit 703 is used to store the step time information of each step length and the flow information of the indicator to be detected of the object to be detected within each step length into the database.
[0129] In a possible implementation, the acquiring unit 701 is further configured to acquire window time information of the second sliding window, where the window time information includes a start time and an end time;
[0130] The processing unit 703 is further configured to store the window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected within the second sliding window into the database.
[0131] In one implementation, the determination unit 702 is further used to determine the survival time of the target data in the database according to the sliding window size, the target data including one or both of step characteristic data and window characteristic data, the step characteristic data being the step time information of each step and the flow information of the indicator to be detected of the object to be detected within each step, and the window characteristic data being the window time information of each sliding window and the flow information of the indicator to be detected of the object to be detected within each sliding window;
[0132] The processing unit 703 is further configured to delete the target data from the database if the storage time of the target data in the database reaches the survival time.
[0133] According to another embodiment of the present application, Figure 7 The various units in the flow determination device shown can be separately or all combined into one or several other units to constitute, or one (some) of the units can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the flow determination device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0134] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 , Figure 4 or Figure 6 A computer program (including program code) for each step involved in part or all of the method shown in the figure is used to construct a computer program (including program code) for each step involved in part or all of the method shown in the figure. Figure 7The flow determination device shown in the embodiment of the present application is used to implement the flow determination method of the embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0135] In the embodiment of the present application, the task to be detected is obtained, and the task to be detected includes an object to be detected and an index to be detected. Through the object to be detected and the index to be detected, the task requirements can be clarified, that is, what results are output. If the index to be detected of the object to be detected has flow at the current time, then according to the current time, the end time of the first sliding window and the sliding window size, the flow sparsity of the index to be detected in the second sliding window corresponding to the object to be detected at the current time is determined, and the first sliding window is the previous sliding window of the second sliding window. This is equivalent to a condition for triggering window calculation. After the previous sliding window completes the flow calculation, when the flow is detected again, the window calculation is triggered, and the flow sparsity of the index to be detected in the second sliding window must first be determined. According to the flow sparsity of the index to be detected in the second sliding window, the target flow calculation strategy is determined, and finally the flow information of the index to be detected in the second sliding window of the object to be detected is determined according to the target flow calculation strategy. In this process, different target flow calculation strategies are selected to perform flow calculation according to the sparsity of the flow of the index to be detected in the second sliding window, which can reduce the calculation amount of the flow information in the sliding window and effectively reduce the calculation cost.
[0136] Based on the above method and device embodiments, the present application embodiment provides a computer device. Figure 8 , Figure 8 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be Figure 1 Server 103 is shown. Figure 8 The computer device shown includes at least a processor 801, an input interface 802, an output interface 803, and a computer-readable storage medium 804. The processor 801, the input interface 802, the output interface 803, and the computer-readable storage medium 804 may be connected via a bus or other means.
[0137] The computer-readable storage medium 804 may be stored in a memory of a computer device. The computer-readable storage medium 804 is used to store a computer program. The computer program includes computer instructions. The processor 801 is used to execute the program instructions stored in the computer-readable storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement a corresponding method flow or a corresponding function.
[0138] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the computer device. In addition, one or more computer instructions suitable for being loaded and executed by the processor are also stored in the storage space. These computer instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.
[0139] In some embodiments, the processor 801 may load and execute one or more computer instructions stored in the computer-readable storage medium 804 to implement the above-mentioned Figure 2 , Figure 4 as well as Figure 6 In a specific implementation, the computer instructions in the computer-readable storage medium 804 are loaded by the processor 801 and execute the following steps:
[0140] Acquire a task to be detected, wherein the task to be detected includes an object to be detected and an indicator to be detected;
[0141] If the indicator to be detected of the object to be detected has traffic at the current time, determine the traffic sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, the end time of the first sliding window, and the size of the sliding window, where the first sliding window is a previous sliding window of the second sliding window;
[0142] Determining a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window;
[0143] The flow information of the indicator to be detected of the object to be detected within the second sliding window is determined according to the target flow calculation strategy.
[0144] In a possible implementation, the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to determine the flow sparseness of the indicator to be detected in the second sliding window corresponding to the current time of the object to be detected according to the current time, the end time of the first sliding window, and the size of the sliding window, and are specifically used to perform the following steps:
[0145] Determine a time interval of empty traffic according to the current time and the end time of the first sliding window;
[0146] Determine a sparse determination threshold according to the sliding window size and a set rule;
[0147] The flow sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time is determined according to the empty flow time interval and the sparse determination threshold.
[0148] In a possible implementation, when the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to determine the target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window, they are specifically used to perform the following steps:
[0149] If the flow sparseness of the indicator to be detected in the second sliding window is sparse, a first flow calculation strategy among the preset multiple flow calculation strategies is determined as the target flow calculation strategy, and the first flow calculation strategy includes: skipping the empty flow time interval and accumulating the flow of the indicator to be detected in the current sliding window;
[0150] If the traffic sparseness of the indicator to be detected in the second sliding window is dense, the second traffic calculation strategy among the preset multiple traffic calculation strategies is determined as the target traffic calculation strategy, and the second traffic calculation strategy includes: integrating the traffic of the indicator to be detected in the previous sliding window, the traffic of the indicator to be detected corresponding to the current time, and the traffic of the indicator to be detected corresponding to the reference time interval; the reference time interval is a time interval composed of the start time of the previous sliding window to the start time of the current sliding window.
[0151] In a possible implementation, the target flow calculation strategy is the first flow calculation strategy, and the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to determine the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy, specifically for performing the following steps:
[0152] Acquire the first step length corresponding to the empty flow of the indicator to be detected in the second sliding window from the database;
[0153] Acquire the flow of the indicator to be detected with a second step length from the database, where the second step length is each step length in the second sliding window except the first step length;
[0154] The flow rate of the indicator to be detected of the second step length is accumulated to obtain the flow rate information of the indicator to be detected of the object to be detected in the second sliding window.
[0155] In a possible implementation, the target flow calculation strategy is the second flow calculation strategy, and the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to determine the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy, specifically for performing the following steps:
[0156] Acquire from a database a first flow rate of the indicator to be detected in the first sliding window, and acquire a second flow rate of the indicator to be detected at the current time;
[0157] Determine a reference time interval according to the start time of the first sliding window and the start time of the second sliding window, and determine a third flow of the indicator to be detected in the reference time interval;
[0158] The first flow, the second flow and the third flow are integrated and processed to obtain flow information of the indicator to be detected of the object to be detected in the second sliding window.
[0159] In one possible implementation, the computer instructions in the computer-readable storage medium 804 are loaded by the processor 801 and are further used to perform the following steps:
[0160] Determine the flow information of the indicator to be detected of each step length of the object to be detected in the second sliding window;
[0161] Acquire step time information of each step in the second sliding window, where the step time information includes a start time and an end time;
[0162] The step time information of each step length and the flow information of the indicator to be detected of the object to be detected within each step length are stored in the database.
[0163] In one possible implementation, the computer instructions in the computer-readable storage medium 804 are loaded by the processor 801 and are further used to perform the following steps:
[0164] Acquire window time information of the second sliding window, where the window time information includes a start time and an end time;
[0165] The window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected within the second sliding window are stored in the database.
[0166] In one implementation, the computer instructions in the computer readable storage medium 804 are loaded by the processor 801 and are further used to perform the following steps:
[0167] Determine the survival time of the target data in the database according to the sliding window size, the target data includes one or both of step characteristic data and window characteristic data, the step characteristic data is the step time information of each step and the flow information of the indicator to be detected of the object to be detected within each step, and the window characteristic data is the window time information of each sliding window and the flow information of the indicator to be detected of the object to be detected within each sliding window;
[0168] If the storage time of the target data in the database reaches the survival time, the target data is deleted from the database.
[0169] In the embodiment of the present application, the task to be detected is obtained, and the task to be detected includes an object to be detected and an index to be detected. Through the object to be detected and the index to be detected, the task requirements can be clarified, that is, what results are output. If the index to be detected of the object to be detected has flow at the current time, then according to the current time, the end time of the first sliding window and the sliding window size, the flow sparsity of the index to be detected in the second sliding window corresponding to the object to be detected at the current time is determined, and the first sliding window is the previous sliding window of the second sliding window. This is equivalent to a condition for triggering window calculation. After the previous sliding window completes the flow calculation, when the flow is detected again, the window calculation is triggered, and the flow sparsity of the index to be detected in the second sliding window must first be determined. According to the flow sparsity of the index to be detected in the second sliding window, the target flow calculation strategy is determined, and finally the flow information of the index to be detected in the second sliding window of the object to be detected is determined according to the target flow calculation strategy. In this process, different target flow calculation strategies are selected to perform flow calculation according to the sparsity of the flow of the index to be detected in the second sliding window, which can reduce the calculation amount of the flow information in the sliding window and effectively reduce the calculation cost.
[0170] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the flow determination method provided in the above-mentioned various optional manners.
[0171] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0172] In the several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, and the division of the modules described above is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for determining flow rate, It is characterized in that The method comprises: Acquire a task to be detected, wherein the task to be detected includes an object to be detected and an indicator to be detected; If the indicator to be detected of the object to be detected has traffic at the current time, determine the traffic sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, the end time of the first sliding window, and the size of the sliding window, where the first sliding window is a previous sliding window of the second sliding window; Determining a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window; The flow information of the indicator to be detected of the object to be detected within the second sliding window is determined according to the target flow calculation strategy.
2. The method according to claim 1, It is characterized in that The determining, according to the current time, the end time of the first sliding window, and the size of the sliding window, the flow sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time includes: Determine a time interval of empty traffic according to the current time and the end time of the first sliding window; Determine a sparse determination threshold according to the sliding window size and a set rule; The flow sparseness of the indicator to be detected of the object to be detected in the second sliding window corresponding to the current time is determined according to the empty flow time interval and the sparse determination threshold.
3. The method according to claim 1 or 2, It is characterized in that The step of determining a target flow calculation strategy according to the flow sparseness of the indicator to be detected in the second sliding window includes: If the flow sparseness of the indicator to be detected in the second sliding window is sparse, a first flow calculation strategy among the preset multiple flow calculation strategies is determined as the target flow calculation strategy, and the first flow calculation strategy includes: skipping the empty flow time interval and accumulating the flow of the indicator to be detected in the current sliding window; If the traffic sparseness of the indicator to be detected in the second sliding window is dense, the second traffic calculation strategy among the preset multiple traffic calculation strategies is determined as the target traffic calculation strategy, and the second traffic calculation strategy includes: integrating the traffic of the indicator to be detected in the previous sliding window, the traffic of the indicator to be detected corresponding to the current time, and the traffic of the indicator to be detected corresponding to the reference time interval; the reference time interval is a time interval composed of the start time of the previous sliding window to the start time of the current sliding window.
4. The method according to claim 3, It is characterized in that The target flow calculation strategy is the first flow calculation strategy, and determining the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy includes: Acquire the first step length corresponding to the empty flow of the indicator to be detected in the second sliding window from the database; Acquire the flow of the indicator to be detected with a second step length from the database, where the second step length is each step length in the second sliding window except the first step length; The flow rate of the indicator to be detected of the second step length is accumulated to obtain the flow rate information of the indicator to be detected of the object to be detected in the second sliding window.
5. The method according to claim 3, It is characterized in that The target flow calculation strategy is the second flow calculation strategy, and determining the flow information of the indicator to be detected of the object to be detected in the second sliding window according to the target flow calculation strategy includes: Acquire from a database a first flow rate of the indicator to be detected in the first sliding window, and acquire a second flow rate of the indicator to be detected at the current time; Determine a reference time interval according to the start time of the first sliding window and the start time of the second sliding window, and determine a third flow of the indicator to be detected in the reference time interval; The first flow, the second flow and the third flow are integrated and processed to obtain flow information of the indicator to be detected of the object to be detected in the second sliding window.
6. The method according to claim 4 or 5, It is characterized in that The method further comprises: Determine the flow information of the indicator to be detected of each step length of the object to be detected in the second sliding window; Acquire step time information of each step in the second sliding window, where the step time information includes a start time and an end time; The step time information of each step length and the flow information of the indicator to be detected of the object to be detected within each step length are stored in the database.
7. The method according to claim 4 or 5, It is characterized in that The method further comprises: Acquire window time information of the second sliding window, where the window time information includes a start time and an end time; The window time information of the second sliding window and the flow information of the indicator to be detected of the object to be detected within the second sliding window are stored in the database.
8. The method according to claim 4 or 5, It is characterized in that The method further comprises: Determine the survival time of the target data in the database according to the sliding window size, the target data includes one or both of step characteristic data and window characteristic data, the step characteristic data is the step time information of each step and the flow information of the indicator to be detected of the object to be detected within each step, and the window characteristic data is the window time information of each sliding window and the flow information of the indicator to be detected of the object to be detected within each sliding window; If the storage time of the target data in the database reaches the survival time, the target data is deleted from the database.
9. A flow rate determination device, It is characterized in that The device comprises: An acquisition unit, used for acquiring a task to be detected, wherein the task to be detected includes an object to be detected and an indicator to be detected; a determining unit, configured to determine, if the indicator to be detected of the object to be detected has flow at the current time, a flow sparseness of the indicator to be detected of the object to be detected in a second sliding window corresponding to the current time according to the current time, an end time of the first sliding window, and a size of the sliding window, wherein the first sliding window is a previous sliding window of the second sliding window; The determining unit is further configured to determine a target flow calculation strategy according to a flow sparseness of the indicator to be detected in the second sliding window; The determining unit is further configured to determine flow information of the indicator to be detected of the object to be detected within the second sliding window according to the target flow calculation strategy.
10. A computer device, It is characterized in that The computer device comprises: a processor adapted to implement one or more computer programs; and, A computer storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by the processor and executing the flow determination method according to any one of claims 1 to 8.
11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores one or more computer programs, and the one or more computer programs are suitable for being loaded by a processor and executing the flow determination method according to any one of claims 1-8.
12. A computer program product, It is characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the flow determination method according to any one of claims 1 to 8.
