Cloud-based statistical report automatic generation system

Abstract

The present application relates to the technical field of report generation, in particular to a cloud-based statistical report automatic generation system, which comprises a path frequency monitoring module, a path access regulation module, a report purpose analysis module, a task priority sorting module and a cloud report generation module. In the present application, the continuity of path access behavior is analyzed and compared with the rate limit, so as to accurately identify the abnormal access frequency. The state blocking and unblocking control mechanism of abnormal paths is introduced, and the dynamic updated path state information is constructed in combination with the access label and time condition, so as to improve the compliance and flexibility of path access. The state conversion result is mapped to the task data item, and the tasks are classified and sorted according to the priority and time constraint. The queue is constructed according to the task execution level, so as to ensure the accurate matching of report generation result and task scheduling, and enhance the data monitoring dimension and report generation effectiveness.

Description

Technical Field

[0001] This invention relates to the field of report generation technology, and in particular to a cloud-based automatic statistical report generation system. Background Technology

[0002] The field of report generation technology involves the organization, statistics, and presentation of structured data. It primarily covers core aspects such as data collection, data analysis, statistical calculation, chart generation, and report layout. Through computer programs, it extracts and processes information from various data sources and outputs visual reports in preset formats. Widely used in enterprise management, financial analysis, and business monitoring, it is a crucial means of data representation in information systems. Traditional report generation refers to using locally deployed data processing tools. In a client-side or LAN environment, data is manually imported, and fixed-rule scripts are written to achieve statistical analysis and report output. This is typically accomplished using the pivot table function of spreadsheet software or database query results. In other words, users manually configure statistical fields, set filtering conditions, summarize calculations, and format the data before outputting it to a document or print format.

[0003] The current statistical report generation process mainly relies on local manual operation. Due to the static nature of client data input, it is difficult to respond in a timely manner in scenarios with frequent data updates or fluctuating access requests. In particular, under multi-path access, it is impossible to effectively identify and preprocess access behavior, resulting in abnormal data frequently being mixed into the statistical results. When the workload is large or the execution priority is unclear, the report content cannot quickly reflect the urgency of the task or the difference in resource requirements. For example, when the access frequency is abnormal, it is still regarded as normal input, which can easily cause the report results to be distorted or the task resource allocation to be unbalanced, affecting the overall decision-making efficiency and system response speed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a cloud-based automatic statistical report generation system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a cloud-based automatic statistical report generation system includes:

[0006] The path frequency monitoring module obtains the path rule ID and timestamp of each access path, arranges the timestamps under the same path ID in order, calculates the length of the continuous access segment with an interval less than the anomaly detection window, counts the number of continuous accesses, compares the maximum number of accesses for each path with the rate limit threshold, and generates an abnormal path log.

[0007] Based on the abnormal path logs, the path access control module extracts the access control label of the abnormal path and the synchronization time of the current NTP. According to the set default blocking period, it constructs a path status table to pair the abnormal path with the remaining blocking time. In the periodic detection, it determines whether the remaining time is 0. If it is 0, it updates the label to the allowed state and generates a compliance status transition audit log.

[0008] The report usage analysis module extracts task data asset catalog items based on the compliance status transition audit log, performs hash lookup, extracts the number of tasks with key business priorities, calculates the cumulative number of tasks within the period, and generates key task indicators.

[0009] Based on the key task indicators, the task priority sorting module extracts the enqueue time of priority task items, calculates the SLA waiting time in combination with the current time, and sorts them in ascending order of priority and waiting time to generate an asset priority queue.

[0010] As a further embodiment of the present invention, the abnormal path log includes a path identifier, a continuous access segment length statistics, and an abnormal access frequency identifier; the path status table includes a path blocking status, remaining blocking time, and a path access control label; the compliance status transition audit log includes access status change records, status change timestamps, and path identity mapping; the key task indicators include the number of tasks of each priority level, task distribution frequency, and task level labels; and the asset priority queue includes task priority sorting, task SLA waiting time, and task enqueue time.

[0011] As a further aspect of the present invention, the path frequency monitoring module includes:

[0012] The path rule extraction submodule obtains the path rule ID and timestamp of each access path, parses the path ID and timestamp of each path request record, uses the path rule ID as a unique identifier, and collects and organizes the corresponding timestamps in the order of receipt to generate a path time series list.

[0013] The time series segmentation submodule calculates the time interval between adjacent timestamps based on the path time series list and the increasing sequence of timestamps corresponding to each path. It filters time periods with intervals smaller than the anomaly detection window as continuous access segments, accumulates the number of accesses for each continuous access segment, obtains multiple continuous access count records under each path, obtains the maximum continuous access frequency value under each path, and generates path access segment statistics results.

[0014] The rate threshold comparison submodule filters the maximum number of consecutive accesses in each path record based on the path access segment statistics, extracts the maximum access frequency value under each path, and compares it with the rate limit threshold item by item to filter the set of path IDs that exceed the threshold and generate abnormal path logs.

[0015] As a further aspect of the present invention, the path access control module includes:

[0016] Based on the abnormal path log, the blocking tag extraction submodule extracts the path identifier field as the key item for retrieval, retrieves the current access control tag of the path, collects the current system time as a unified reference time, and integrates them to generate an access status tag list.

[0017] The blocking status mapping submodule, based on the access status tag list and according to the default blocking period, compares the NTP time in each path record with the first blocking time of the abnormal path, calculates the remaining blocking duration at the current moment, and if the duration is greater than 0 seconds, retains the current path's blocking status unchanged. At the same time, it constructs a mapping relationship between the path and the remaining blocking duration using the path ID as an index, and generates a path blocking status table.

[0018] The periodic label update submodule periodically traverses all current entries in the path blocking status table, determines whether the remaining blocking time field of each path is less than or equal to 0 seconds, and if the condition is met, updates the access control label of the corresponding path to allow, records the update event, and generates a compliance status transition audit log.

[0019] As a further aspect of the present invention, the report usage parsing module includes:

[0020] The directory data extraction submodule, based on the compliance status transition audit log, decomposes each log record into fields, locates the path identification information and corresponding access control status change fields recorded in the fields, extracts the path identification as the task data identification item, and extracts the task execution time point within the period window by combining the change time field associated in the log, and summarizes and generates the task data directory structure.

[0021] The priority mapping submodule uses the path ID of each record as the key value to perform a hash match lookup with the ITIL service directory priority mapping table based on the task data directory structure. It also calculates the task completion delay by combining the task execution time field, calculates the number of valid tasks in each task level, and generates task quantity statistics.

[0022] The task indicator statistics submodule summarizes the cumulative number of valid tasks under each level within a period based on the task quantity statistics results. The statistical granularity unit is set to hour, and the number of occurrences and task proportions of Level 1 to Level 4 tasks are calculated in each hour to generate key task indicators.

[0023] As a further aspect of the present invention, the task priority sorting module includes:

[0024] The entry time extraction submodule extracts the creation time field recorded in each task path according to the key task indicators and classifies them by level. It also parses whether there are relay or transfer records in the task field, constructs a task entry detail table, and generates a task entry time sequence.

[0025] The waiting time calculation submodule calculates the cumulative waiting time of each task from enqueuing to the present based on the task enqueuing time sequence and the current system time, and calculates and outputs the task priority ranking value of each task unit at the current time.

[0026] The priority queue generation submodule sorts the task sequence in ascending order according to the task priority ranking value. The sorting process is nested according to the task level priority strategy. Within the same level, tasks are arranged according to the ranking value. Between levels, priority dimensions are arranged in order from Level 1 to Level 4 to generate an asset priority queue.

[0027] As a further aspect of the present invention, the system further includes:

[0028] The cloud-based report generation module detects the current schedulable resource indicators, including the number of available CPU cores, available memory capacity, and queue depth, based on the asset priority queue, and sequentially completes data aggregation, field matching, and table structure generation to output a cloud-based statistical report.

[0029] The cloud-based statistical reports include scheduling resource usage, task execution matching degree, and report field structure.

[0030] As a further aspect of the present invention, the cloud-based report generation module includes:

[0031] The resource status detection submodule obtains the path fields of all tasks to be scheduled in the asset priority queue, and sequentially detects the current schedulable resource indicators of the system, including the number of available CPU cores, available memory capacity and remaining depth of the scheduling queue, summarizes and constructs a resource status parameter set, and generates a resource utilization parameter group.

[0032] The field mapping calculation submodule performs scheduling adaptation analysis on the correspondence between task path ID and current resource status based on the resource utilization parameter group and the task field in the asset priority queue, analyzes the matching degree between task and current resource status, and establishes field adaptation results.

[0033] The report structure output submodule sorts the tasks according to the field adaptation results, extracts key fields for all tasks in the matching order, fills them in a structured manner, constructs a standardized output format, and summarizes and integrates them to generate a cloud-based statistical report.

[0034] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0035] In this invention, by analyzing the continuity of path access behavior and comparing rate limits, it is possible to accurately identify abnormal access frequencies. A state blocking and unblocking control mechanism for abnormal paths is introduced. By combining access tags and time conditions, dynamically updated path status information is constructed, improving the compliance and flexibility of path access. Then, the state transition results are mapped to task data items, and tasks are classified and sorted according to priority and time constraints. Furthermore, queues are constructed based on task execution levels, thereby enabling the data processing flow to have linkage, automation, and task urgency identification capabilities. This ensures that the report generation results and task scheduling are accurately matched, while realizing closed-loop control of the entire process from access behavior analysis to task indicator extraction to resource scheduling, enhancing the dimensions of data monitoring and the effectiveness of report generation. Attached Figure Description

[0036] Figure 1 This is a system flowchart of the present invention;

[0037] Figure 2 This is a flowchart of the path frequency monitoring module of the present invention;

[0038] Figure 3 This is a flowchart of the path access control module of the present invention;

[0039] Figure 4 This is a flowchart of the report usage analysis module of the present invention;

[0040] Figure 5 This is a flowchart of the task priority sorting module of the present invention;

[0041] Figure 6 This is a flowchart of the cloud-based report generation module of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0043] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0044] Please see Figure 1 The cloud-based automatic statistical report generation system includes:

[0045] The path frequency monitoring module obtains the AWSWAF path rule ID (Web Application Firewall path identifier format) and ISO8601 timestamp for each access path, arranges the timestamps under the same path ID in order, calculates the length of continuous access segments with an interval less than the OWASF PCRS3 anomaly detection window (60 seconds, the 60-second detection window defined by the Open Web Application Security Project core rule set), counts the number of consecutive accesses, compares the maximum number of accesses for each path with the rate limit threshold (15 times / minute), and generates anomaly path logs.

[0046] The path access control module extracts the ISO27001 access control label (access status label defined by the information security management standard) of the abnormal path from the abnormal path log and synchronizes it with the current NTP time. Based on the set AWSShield Advanced default blocking period (the default automatic blocking period of 5 minutes for DDoS protection service, 300 seconds), it constructs an SNMPMIB path status table (Simple Network Management Protocol Management Information Base format) to pair the abnormal path with the remaining blocking duration. In the periodic detection, it checks whether the remaining duration is 0. If it is 0, it updates the label to the allowed status and generates a compliance status transition audit log.

[0047] The report usage analysis module extracts task data asset catalog items from the compliance status transition audit log, performs a hash lookup with the ITIL service catalog priority mapping table (ITIL infrastructure library priority classification standard), extracts the number of tasks with key business priorities (Level 1-4), calculates the cumulative number of tasks within the statistical period, and generates key task indicators.

[0048] Level 1 (Critical Task): ≤15 minutes;

[0049] Level 2 (Important Task): ≤1 hour;

[0050] Level 3 (Regular Task): ≤4 hours;

[0051] Level 4 (low priority): ≤24 hours.

[0052] The task priority module extracts the enqueue time of priority tasks based on key task indicators, calculates the SLA waiting time in combination with the current time, and generates an asset priority queue by sorting them in ascending order of priority and waiting time.

[0053] The cloud-based report generation module detects currently schedulable resource metrics based on the asset priority queue, including the number of available CPU cores, available memory capacity, and queue depth. It then performs data aggregation, field matching, and table structure generation in sequence, outputting cloud-based statistical reports.

[0054] The abnormal path log includes path identifier, continuous access segment length statistics, and abnormal access frequency indicators. The path status table includes path blocking status, remaining blocking time, and path access control labels. The compliance status transition audit log includes access status change records, status change timestamps, and path identity mapping. Key task indicators include the number of tasks of each priority level, task distribution frequency, and task level labels. The asset priority queue includes task priority sorting, task SLA waiting time, and task enqueue time. The cloud statistical reports include scheduling resource usage, task execution matching degree, and report field structure.

[0055] Please see Figure 2 The path frequency monitoring module includes:

[0056] The path rule extraction submodule obtains the path rule ID and timestamp of each access path, parses the path ID and timestamp of each path request record, uses the path rule ID as a unique identifier, and collects and organizes the corresponding timestamps in the order of receipt to generate a path time series list.

[0057] To obtain the AWSWAF path rule ID and ISO8601 timestamp for each access path, the HTTP request logs are first preliminarily parsed to extract the URI path field and request time field. The URI path is standardized and translated by matching the regular expression set in the WAF rule to form a unique path rule identifier. The WAF path rule ID is often matched using the URI structure as a template. For example, the request / api / user / info can be mapped to the rule ID: api_user_wildcard through the rule / api / user / *. This mapping is quickly converted using a hash mapping table. The extracted timestamps must be uniformly in the ISO8601 standard format, such as "2024-06-05T08:12:54Z", for easy subsequent sorting. The parsed path rule ID and timestamp are then used as a tuple to form a preliminary record. The queue, such as [(api_user_wildcard, 2024-06-05T08:12:54Z), (api_user_wildcard, 2024-06-05T08:13:15Z), ...], is then clustered and classified according to the path rule ID as the key. All access records belonging to the same rule ID are grouped together. For example, the path access records with the rule ID api_user_wildcard appear at 08:12:54, 08:13:15, and 08:13:42 respectively. After initial merging, a time series data structure with the path ID as the key is formed. For each time series, an ascending sort operation is performed by the timestamp to ensure that the timestamp sequence under each path is strictly ascending from the beginning to the end, thus forming a standardized path time series list. This is further organized and stored in JSON structure. Some data can be seen in the table below.

[0058] Table 1: Sample Time Series Paths

[0059] ;

[0060] As shown in Table 1, accesses at different times are aggregated and sorted according to the path rule ID to ensure that each path request sequence is continuous and searchable, and finally a path time series list is obtained.

[0061] The time series segmentation submodule calculates the time interval between adjacent timestamps based on the path time series list and the increasing sequence of timestamps corresponding to each path. It filters time periods with intervals smaller than the anomaly detection window as continuous access segments, accumulates the number of accesses for each continuous access segment, obtains multiple continuous access count records under each path, obtains the maximum continuous access frequency value under each path, and generates path access segment statistics results.

[0062] Based on the path time series list, and the increasing sequence of timestamps corresponding to each path, the interval between each timestamp and its subsequent timestamps is first determined. This is calculated by subtracting the current timestamp from the subsequent timestamp to obtain the time difference in seconds, using UTC as the time base. If two timestamps are 08:12:54 and 08:13:15, the interval is 21 seconds. All consecutive timestamp pairs are iterated over. If the time interval is less than 60 seconds (OWASPCRS3 anomaly detection window), it is marked as a continuous access segment. The start and end positions are recorded, and the number of accesses within the segment is accumulated. For example, if a segment contains timestamps 08:12:54, 08:13:15, and 08:13:42, since the interval between each pair is less than 60 seconds, the accumulated access count is 3, and this value is recorded. If an access segment occurs at 08:12:54 and 08:14:10, with an interval exceeding 60 seconds, a new segment is started, forming access counts for different segments. After traversing the entire path, the number of consecutive access segments under each path is summarized. The path rule ID is used as an index to record all its subordinate consecutive segments and their respective access counts. Then, the maximum value is extracted from the list of consecutive segment access counts under each path to determine the highest access frequency reached by the path in any consecutive segment. For example, the segment count list under the path api_user_wildcard is [3, 2, 4], with a maximum value of 4 times. This value indicates that the user accessed the path a maximum of 4 times in a certain consecutive time period. Finally, a mapping structure is established between the maximum access frequency value and the path ID to generate the path access segment statistics.

[0063] The rate threshold comparison submodule filters the maximum number of consecutive accesses in each path record based on the path access segment statistics, extracts the maximum access frequency value under each path, and compares it with the rate limit threshold item by item. It then filters the set of path IDs that exceed the threshold and generates abnormal path logs.

[0064] Based on the path access segment statistics, the maximum number of consecutive accesses in each path record is filtered. First, a rate limit threshold of 15 times / minute is set, based on the system's early warning mechanism. This threshold references the general request rate boundary defined by OWASP. After internal testing and operational system evaluation, 15 times / minute was selected as the initial benchmark. The maximum number of accesses in the path access segment statistics is compared. If the maximum number of accesses is greater than 15, it is considered abnormal path access behavior. For example, if the maximum access frequency for path ID api_user_wildcard is 17 times, then 17 > 15, and it is marked as abnormal. If the access frequency for path ID / api / message / push is 13 times, then it is below the threshold and not marked. All path IDs exceeding the threshold are aggregated to construct an abnormal path list, and log records are output, such as the record item: "Path ID: api_user_wildcard, Maximum Frequency: 17, Exceeds Threshold". Finally, all abnormal path information is summarized to generate an abnormal path log.

[0065] Please see Figure 3 The path access control module includes:

[0066] The blocking tag extraction submodule extracts the path identifier field as the key item for retrieval based on the abnormal path log, retrieves the current access control tag of the path, collects the current system time as a unified reference time, and integrates and generates a list of access status tags.

[0067] Based on the abnormal path logs, each abnormal access event in the log is first read one by one, and the path identifier ID stored in the field is extracted as a query index. This index field is usually a string type, for example, the path " / api / user / login" corresponds to the identifier "api_user_login". Next, the access control policy database is called, and the corresponding field is the ISO27001 access control label. This field indicates the current access permission status of the path, and the label value is fixed as "blocked" or "allowed". This label is the initial security set by the system policy engine after the path is determined to be an over-restricted access path based on the abnormal frequency identification process in the previous stage. The attribute identifier is then used to collect the system's current timestamp via the NTP time synchronization module. The format is uniformly ISO8601 standard, such as "2024-06-05T08:36:42Z", which is used for linkage with subsequent unblocking cycle judgment. Then, the three parameters extracted above, path ID, access control tag and timestamp, are integrated into a unified structure record. The structure example is as follows: path ID = "api_user_login", access tag = "blocked", collection time = "2024-06-05T08:36:42Z". This type of structure information is summarized in a list. Each item in the list is a snapshot of the security status of an independent path. Some examples are listed in Table 2.

[0068] Table 2 Sample Table of Path Access Status Labels:

[0069] ;

[0070] As shown in Table 2, path access control labels are matched and archived using standard synchronization time, which provides a basis for subsequent status judgment and label updates, and finally generates an access status label list.

[0071] The blocking status mapping submodule is based on the access status label list and the default blocking period. It compares the NTP time in each path record with the first blocking time of the abnormal path, calculates the remaining blocking time at the current moment, and if the duration is greater than 0 seconds, it retains the current path's blocking status. At the same time, it constructs a mapping relationship between the path and the remaining blocking time using the path ID as an index, and generates a path blocking status table.

[0072] Based on the access status tag list, the path ID associated with each record is retrieved to obtain the time record of the first time the path was first blocked. This time value usually comes from the database of previous block trigger records and is also in UTC format, such as "2024-06-05T08:31:12Z". Then, a difference operation is performed between this time and the current time field to calculate the duration the path has been blocked since the initial block. This value is expressed in seconds. For example, if the current time is 08:36:42 and the block time is 08:31:12, the time difference is 330 seconds. This difference is further compared with the preset AWSShield Advanced blocking threshold of 30. The comparison starts at 0 seconds. If the difference is less than 300 seconds, the remaining blocking time for the path is calculated as 300 minus the current time difference, i.e., 300-330=-30, indicating that the blocking time has expired. Conversely, if the difference is 270 seconds, the remaining time is 30 seconds, and the blocking is still in progress. Then, the path ID, original access tag, remaining blocking time, and record update time are combined to construct a path status record in SNMP structure format. This structure must meet the MIB format field standard, and the field naming convention must be uniform. All unit conversions and field values ​​are processed by program logic and then output and organized. Finally, the blocking information of each path is combined into a structured form and summarized into a path blocking status table.

[0073] The periodic label update submodule periodically traverses all current entries in the path blocking status table, determines whether the remaining blocking time field of each path is less than or equal to 0 seconds, and if the condition is met, updates the access control label of the corresponding path to allow, records the update event, and generates a compliance status transition audit log.

[0074] Based on the path blocking status table, the system sets a timed detection cycle, performing a table entry traversal operation every 60 seconds. The system checks the remaining blocking time field in each record, judging whether the value is less than or equal to 0 seconds. If the value is negative or 0, it indicates the current blocking time has ended. The system then updates the ISO27001 access control label field corresponding to that path, resetting it from "Blocked" to "Allowed." This operation simultaneously updates the label status field and the label update time field. For example, the original status of the path "api_user_login" is "Blocked," with a remaining time of -30 seconds. After the periodic detection, the system performs a label update, changing it to "Allowed," with the change time recorded as "2024-06-05T08:36:42Z." The path ID, the label before the change, the label after the change, and the update time field are then integrated into a label change audit record. All audit records are summarized to form a form data structure, serving as the audit tracing basis for path label adjustments, ultimately generating a compliance status transition audit log.

[0075] Please see Figure 4 The report usage analysis module includes:

[0076] The directory data extraction submodule is based on the compliance status transition audit log. It breaks down each log record into fields, locates the path identification information and corresponding access control status change fields, extracts the path identification as the task data identification item, and extracts the task execution time point within the period window by combining the change time field associated in the log, and summarizes and generates the task data directory structure.

[0077] Based on the compliance status transition audit logs, the path ID and change time fields are first extracted from the logs. Each path record is then categorized into the corresponding business system directory at the task granularity. The path structure is decomposed by separating fields with a forward slash " / ". The first-level field serves as the system module identifier, the second-level field as the function category identifier, and the last-level field as the operation behavior identifier. For example, the path " / finance / pay / check" is parsed as the module "finance", function "pay", and behavior "check". Next, the timestamp field is extracted from the logs and uniformly converted to the ISO8601 time standard format, such as "2024-06-05T10:15:07Z". Combined with the path structure information, a three-tuple structure task record item is constructed, with the fields "Path ID", "Directory Structure Item", and "Execution Time". All extracted items are aggregated into a data table, as shown below:

[0078] Table 3: Task Data Directory Structure Table

[0079] ;

[0080] As shown in Table 3, each task path is structurally divided and time-extracted based on its appearance in the audit log, establishing a clear task identification structure and directory attribute mapping, and finally generating a task data directory structure.

[0081] The priority mapping submodule, based on the task data directory structure, uses the path ID of each record as the key to perform a hash match lookup with the ITIL service directory priority mapping table, and calculates the task completion delay by combining the task execution time field with the following formula:

[0082] ;

[0083] Calculate the number of valid tasks in each task level and generate task count statistics. Indicates task level The effective quantity coefficient of the task is as follows. Indicates the task status update time. Indicates the task trigger time. The time delay values ​​are fixed priority values ​​for each level (Level 1 = 900 seconds, Level 2 = 3600 seconds, Level 3 = 14400 seconds, Level 4 = 86400 seconds). This indicates the periodic task distribution density corresponding to the task level. This represents the variance of execution time in the graded task;

[0084] Based on the task data directory structure, each record undergoes mapping and matching. First, the "Behavior" field in the table is used as the primary index field, combined with the "Function Category" field for a double lookup, corresponding to the level classification information in the ITIL priority mapping table. If the behavior field is "submit", it maps to Level 1; if it's "certify", it maps to Level 2; "check" to Level 3; and "request" to Level 4. After priority identification, the time interval from task creation to change is calculated based on the task path record time and its subsequent status change time. For example, if the path " / risk / audit / certify" starts at 2024-06-05T10:12:33 and ends at 2024-06-05T10:24:01, the interval is 688 seconds, mapping to a Level 2 task. Comparing this to the maximum allowed time of 3600 seconds for a Level 2 task, which is less than the threshold of 688 < 3600, the task is considered valid. Based on this, the following parameter values ​​are then introduced:

[0085] seconds (convert to seconds);

[0086] Second;

[0087] Second;

[0088] Seconds (Level 2);

[0089] (Level 2 task cycle density);

[0090] (Level 2 task time variance, seconds²);

[0091] Substitute the values ​​into the calculation formula as follows:

[0092] ;

[0093] The result indicates that the effective count value of Level 2 tasks after weighted calculation within this period is approximately 95.29. If this operation is repeated for multiple tasks, the effective number of tasks at this level can be counted, and the final task count result can be obtained.

[0094] The number of valid tasks in each task level refers to the number of task instances that meet the level response requirements after being classified by task level within a set period of time, and after judging their execution time and priority response time limit, combined with the task quantity density and stability adjustment. This number not only reflects the actual completion status of the task level in the current period, but also reflects the pressure and processing effect of the task level in terms of operation and maintenance support, service capacity and timeliness control to a certain extent. It is an important statistical indicator for measuring the compliance of critical task scheduling, the rationality of service load distribution and resource matching efficiency.

[0095] The formula's computational logic is designed to integrate four dimensions: actual task time consumption, expected response level, task quantity density, and task stability, to construct a weighted evaluation coefficient. This coefficient measures the effectiveness and regulatory pressure of each task level within the cycle. The numerator part... This represents the sum of the actual task time and the expected time limit for the assigned level. It reflects whether the task time is close to or exceeds its level threshold. If the task time is longer, the overall value increases, reflecting increased execution pressure. The denominator is... This takes into account the quantity density of tasks of this level within the current cycle. variance of task duration By using the square root operation, the variance is compressed to an order of magnitude similar to the density, thus reflecting the stability of the task distribution in the evaluation. If the task duration fluctuates greatly, then... An increase in the denominator will ultimately lower the assessed value. This indicates that the task at this level is highly volatile and more difficult to control. The structure uses absolute value calculations as a whole to ensure that the evaluation value is always non-negative, which facilitates horizontal comparison between different levels.

[0096] The task indicator statistics submodule summarizes the cumulative number of valid tasks under each level within a period based on the task quantity statistics results. The statistical granularity unit is set to hour, and the number of occurrences and task proportions of Level 1 to Level 4 tasks within an hour are calculated to generate key task indicators.

[0097] Based on the task quantity statistics, the frequency of occurrence of each type of task within a one-hour period was categorized and summarized. The period was set from 10:00:00 to 11:00:00 on June 5, 2024. The number of tasks of different levels within the statistical period was as follows: Level 1: 22 tasks, Level 2: 47 tasks, Level 3: 91 tasks, and Level 4: 15 tasks, for a total of 175 tasks. The percentage of each level was calculated as follows: Level 1: 12.6%, Level 2: 26.9%, Level 3: 52%, and Level 4: 8.5%. Subsequently, the statistical dimension was refined to a 15-minute granularity to obtain the task quantity trend, as shown in the following statistics:

[0098] Table 4: Statistical Results of Key Tasks

[0099] ;

[0100] As shown in Table 4, the statistical results of different levels of tasks within a specified period clearly represent the number and distribution trend of task execution. This table structure can serve as the basic record for task management and capacity monitoring, and ultimately generate key task indicators.

[0101] Please see Figure 5 The task prioritization module includes:

[0102] The entry time extraction submodule extracts the creation time field recorded in each task path based on key task indicators and classifies them by level. It also parses whether there are relay or transfer records in the task field, constructs a task entry details table, and generates a task entry time series.

[0103] Based on the path level and behavior type recorded in the key task indicators, the first step is to extract the task's enqueue time field. This field indicates the time when the task is officially included in the processing queue. During processing, it is necessary to determine whether multiple task records belong to the same task chain. If multiple updates exist, the initial creation time, rather than the update time, should be identified as the enqueue time. For example, if the path " / ops / task / submit" appears twice in the log, at times 2024-06-05T10:10:12 and 2024-06-05T10:35:44 respectively, the former is the valid enqueue time. In the field extraction, all path records are standardized using the ISO8601 time format to ensure comparability between time fields. Next, the module, function, and behavior fields are extracted for each path in sequence. The path structure is divided by a forward slash " / ", and the meaning of the fields is locked by their position order. For example, the path " / finance / credit / apply" is parsed as the module being finance, the function being credit, and the behavior being apply. Subsequently, a task data detail table is constructed and the field data is filled in row by row, resulting in the following structure:

[0104] Table 5 Task Entry Details:

[0105] ;

[0106] As shown in Table 5, by decomposing the path field and extracting the queuing time field, and combining the task level to form a structured queuing record, a task queuing time sequence is established.

[0107] The waiting time calculation submodule calculates the cumulative waiting time for each task from its enqueue entry time sequence to the current system time, using the following formula:

[0108] ;

[0109] Calculate and output the task priority ranking value of each task unit at the current time, where, Indicates the first Each task is at the level The sorted values ​​below, This is the current waiting time. This represents the historical average response time for this task. The historical response incentive coefficient, To handle the complexity of the function, For level Fixed SLA response threshold, Indicates the multi-factor interference degree of the path. The influence coefficient of each interfering factor;

[0110] Based on the task enqueue time series, the current system time is used as a timestamp reference value. The timestamps are then converted to second-level values ​​and the difference is calculated. For example, if the current time is 2024-06-05T11:00:00, corresponding to 39600 seconds, and the task enqueue time is 2024-06-05T10:10:12, or 36612 seconds, the difference is 2988 seconds, indicating a task waiting time of 2988 seconds. Subsequently, the historical average response time of the task is extracted. Functional processing complexity Level response threshold Path interference factor And set the historical response incentive coefficient. With interference factor weighting coefficient The values ​​for each parameter are as follows:

[0111] , , , , , , ;

[0112] Substitute the values ​​and calculate as follows:

[0113] ;

[0114] The results indicate that the task ranking value is approximately 1.305, and a lower ranking value indicates a higher urgency of task processing.

[0115] The task ranking value is a numerical expression that quantifies the urgency of each task under the current system scheduling status. This value integrates the task's cumulative waiting time in the queue, historical response performance, functional complexity, SLA response threshold of the task level, and the weighted influence of multi-dimensional interference factors. The smaller the result obtained after comprehensive calculation, the greater the current waiting pressure, the tighter the response expectation, and the more stable the path of the task. Therefore, it should be scheduled and executed with priority among tasks of the same level. Conversely, the larger the ranking value, the more tolerable the task is, or the path is complex and the influencing factors are strong. It does not need to be prioritized in the short term. Therefore, the task ranking value is the core evaluation basis for the scheduling system to make decisions on the order of task execution and resource allocation.

[0116] Historical response incentive coefficient The value is set based on the response time volatility of the task in past scheduling cycles. Specifically, it is calculated by collecting historical task data that matches the current task level and calculating the standard deviation of its response time. Then, compared with the SLA threshold of that level. Forming normalized volatility and will In the linear mapping interval Internal control mapping to If the standard deviation of the response time in a certain Level 2 task sample set is 288 seconds, and ,but If it falls into the lower segment of the mapping range, it will eventually correspond to... This value indicates that the task response history is relatively stable but has slight fluctuations; interference factor weighting coefficient The settings are based on the statistical contribution ratio of various task execution bottlenecks in abnormal tasks. First, the interference factors involved in the task path are identified. Factors such as CPU processing bottlenecks, I / O blocking, and authentication delays are identified. The frequency of each factor in failed or timed-out tasks is then statistically analyzed, and each frequency is normalized to the total frequency, ensuring that the sum of the weights of all interfering factors is 1. For example, in nearly 100 abnormal tasks, if a CPU processing bottleneck occurs 30 times and authentication-related delays occur 70 times, then the normalized weights would be... , Both reflect the influence weight of such interference factors and the actual sharing ratio of ranking resistance in the ranking value evaluation, ensuring that they can be used to construct task path stability regulation factors.

[0117] The formula's operational logic aims to comprehensively reflect the processing priority of tasks in the queue; the numerator is determined by the task's current waiting time. Its historical average response time With response excitation coefficient The product and the complexity of functional processing The formula is composed of the square root of the complexity, where the waiting time reflects the urgency of the task being stuck in the queue, and the historical average response time, weighted by the incentive coefficient, reflects the expected response pressure of the task execution. The sum of these two is then subtracted from the square root of the complexity. This square root operation is used to mitigate the extreme amplification of the sorting effect of high-complexity tasks, bringing it to the same order of magnitude as the time dimension to avoid imbalance. The denominator is determined by the pre-set SLA response threshold. The set of interference degrees is composed of a weighted sum of the interference degrees from multiple path factors. This represents the various factors affecting the stability of the task path, multiplied by their respective weighting coefficients. This reflects the degree of influence of the factor during scheduling. The summation forms the control factor for task path stability. This factor is added to the SLA threshold to construct the total ranking resistance factor. The absolute value of the overall formula is then taken to avoid the numerator being negative, which would cause the ranking value to deviate unexpectedly. The final ranking value is used to assess the current scheduling urgency of the task.

[0118] The priority queue generation submodule sorts the task sequence in ascending order according to the task priority ranking value. The sorting process is nested according to the task level priority strategy. Within the same level, the tasks are sorted according to the ranking value. Between levels, the priority dimensions are arranged in the order of Level1 to Level4 to generate the asset priority queue.

[0119] Based on task priority ranking, all tasks are nested and arranged in ascending order of priority level and ranking value, from Level 1 to Level 4. Within each Level, tasks are sorted in ascending order of ranking value. If two tasks have ranking values ​​of 0.96 and 1.17 respectively, the former is ranked first. This forms a structured queue record with fields including path ID, task level, ranking value, and wait time. All fields are output sequentially to form a sorted queue list, as follows:

[0120] Table 6 Task Priority Queue Sorting Table:

[0121] ;

[0122] As shown in Table 6, the task priority queue has been arranged in ascending order of level and sorting value for the scheduling system to process sequentially, and finally an asset priority queue is established.

[0123] Please see Figure 6 The cloud-based report generation module includes:

[0124] The resource status detection submodule obtains the path field of all tasks to be scheduled in the asset priority queue, and sequentially detects the current schedulable resource indicators of the system, including the number of available CPU cores, available memory capacity, and remaining depth of the scheduling queue. It then summarizes and constructs a resource status parameter set to generate a resource utilization parameter group.

[0125] Based on the step-by-step parsing of each task path field in the asset priority queue, the field structure in the task path ID is first broken down to confirm the processing module and type to which the task belongs. Then, the connected scheduling resource monitoring system is invoked to collect the current available CPU cores, remaining memory capacity, and task queue depth configured by the scheduler. The number of CPU cores represents the scheduling core's scheduling capacity. This is determined by querying the " / proc / stat" interface to parse the current number of idle processing cores. For example, if there are 16 total cores and 10 running cores, then there are 6 idle cores. Memory capacity is read from " / proc / meminfo" or real-time cache data from the resource monitoring module. A current idle capacity of 22.7GB indicates strong remaining resource capacity. Queue depth is the difference between task enqueueing and dequeueing. For example, if the maximum capacity is 200 tasks and 147 tasks are currently queued, then the remaining depth is 53. The three resource indicators collected need to be recorded in a unified format with consistent sampling time. Subsequently, each task path field is paired with the above indicators to form a pairing record of task and system resource status, as follows:

[0126] Table 7 Task Resource Status Matching Table:

[0127] ;

[0128] As shown in Table 7, the system resource status and task path have been successfully matched to obtain the resource utilization parameter set.

[0129] The field mapping calculation submodule performs scheduling adaptation analysis on the correspondence between task path ID and current resource status based on the resource utilization parameter group and the task field in the asset priority queue. It analyzes the matching degree between tasks and current resource status and establishes field adaptation results.

[0130] Based on the resource utilization parameter group and the task fields in the asset priority queue, the processing requirements of each task are extracted item by item, including the estimated CPU core requirements, memory consumption value, and processing type field. If the task path is " / risk / log / submit", historical records show that this type of task has an average CPU core usage of 2.3 cores and an average memory usage of 5.6GB during normal operation. These fields are then used as reference values ​​for task resource consumption and compared with the currently available resource values. If the current idle CPU cores are 6 cores and memory is 22.7GB, then the resource suitability of this task is high. Subsequently, all task records in the asset queue are processed sequentially, the task level field is extracted, and a resource requirement priority multiplier is assigned. For example, Level 1 task... The source demand multiplier is 1.3, Level 2 is 1.1, Level 3 is 1.0, and Level 4 is 0.9. This is used to correct the resource consumption value of the task itself, so as to reflect the amplified performance of the actual demand when allocating resources for high-level tasks. Then, according to the task behavior field, if it is a processing operation such as "submit" or "verify", a task resource adaptation correction value is added. For example, if the submit behavior causes the largest fluctuation in memory consumption, an additional correction amount of 0.5GB is added and superimposed on the aforementioned correction value. After matching the final resource adaptation ratio of memory and CPU for each task, the adaptation degree is extracted into three levels: excellent, medium, and poor according to standard rules. Finally, the task status interval level is marked according to the adaptation matching result, and the field adaptation result is obtained.

[0131] The report structure output submodule sorts the tasks according to the field adaptation results, extracts key fields for all tasks in the matching order, fills them in a structured manner, builds a standardized output format, summarizes and integrates them, and generates cloud statistical reports.

[0132] Based on the resource adaptation levels assigned by the field adaptation results, all tasks are first sorted from best to worst, with top-tier tasks reported first, followed by medium-tier tasks, and finally poor-tier tasks. Within each level, tasks are further sorted in ascending order based on their priority field to form a complete scheduling execution sequence. Then, the task path ID, task level, resource requirement field, and adaptation level value for each task are extracted and bound to the structure fields. These fields are then filled into the corresponding field rows of the report structure template. The template field order is: Task ID, Level, CPU Core Requirement, Memory Requirement, Adaptation Level, and Resource Status Identifier. For example, if the task " / risk / log / submit" is a Level 1 task with a CPU requirement of 2.3 cores, a memory requirement of 5.6GB, and a resource adaptation level of best, the corresponding fields would be " / risk / log / submit, Level 1, 2.3, 5.6, Best, Scheduled". After filling all tasks with these fields, the cloud data table structure is constructed as follows:

[0133] Table 8 Cloud Report Output Structure:

[0134] ;

[0135] As shown in Table 8, all fields have been formatted, populated, and sorted to suit the task, and a cloud-based statistical report has been created.

[0136] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A cloud-based automatic statistical report generation system, characterized in that, The system includes: The path frequency monitoring module obtains the path rule ID and timestamp of each access path, arranges the timestamps under the same path ID in order, calculates the length of the continuous access segment with an interval less than the anomaly detection window, counts the number of continuous accesses, compares the maximum number of accesses for each path with the rate limit threshold, and generates an abnormal path log. Based on the abnormal path logs, the path access control module extracts the access control label of the abnormal path and the synchronization time of the current NTP. According to the set default blocking period, it constructs a path status table to pair the abnormal path with the remaining blocking time. In the periodic detection, it determines whether the remaining time is 0. If it is 0, it updates the label to the allowed state and generates a compliance status transition audit log. The report usage analysis module extracts task data asset catalog items based on the compliance status transition audit log, performs hash lookup, extracts the number of tasks with key business priorities, calculates the cumulative number of tasks within the period, and generates key task indicators. The report usage analysis module includes: The directory data extraction submodule, based on the compliance status transition audit log, decomposes each log record into fields, locates the path identification information and corresponding access control status change fields recorded in the fields, extracts the path identification as the task data identification item, and extracts the task execution time point within the period window by combining the change time field associated in the log, and summarizes and generates the task data directory structure. The priority mapping submodule uses the path ID of each record as the key value to perform a hash match lookup with the ITIL service directory priority mapping table based on the task data directory structure. It also calculates the task completion delay by combining the task execution time field, calculates the number of valid tasks in each task level, and generates task quantity statistics. The task indicator statistics submodule summarizes the cumulative number of effective tasks under each level of task within a period based on the task quantity statistics results. The statistical granularity unit is set to hour, and the number of occurrences and task proportions of Level 1 to Level 4 tasks within an hour are calculated to generate key task indicators. Based on the key task indicators, the task priority module extracts the enqueue time of priority task items, calculates the SLA waiting time in combination with the current time, and generates an asset priority queue by sorting them in ascending order of priority and waiting time. The cloud-based report generation module detects the current schedulable resource indicators, including the number of available CPU cores, available memory capacity, and queue depth, based on the asset priority queue, and then performs data aggregation, field matching, and table structure generation in sequence to output a cloud-based statistical report.

2. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The abnormal path log includes path identifier, continuous access segment length statistics, and abnormal access frequency identifier. The path status table includes path blocking status, remaining blocking time, and path access control label. The compliance status transition audit log includes access status change records, status change timestamps, and path identity mapping. The key task indicators include the number of tasks of each priority level, task distribution frequency, and task level label. The asset priority queue includes task priority sorting, task SLA waiting time, and task enqueue time.

3. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The path frequency monitoring module includes: The path rule extraction submodule obtains the path rule ID and timestamp of each access path, parses the path ID and timestamp of each path request record, uses the path rule ID as a unique identifier, and collects and organizes the corresponding timestamps in the order of receipt to generate a path time series list. The time series segmentation submodule calculates the time interval between adjacent timestamps based on the path time series list and the increasing sequence of timestamps corresponding to each path. It filters time periods with intervals smaller than the anomaly detection window as continuous access segments, accumulates the number of accesses for each continuous access segment, obtains multiple continuous access count records under each path, obtains the maximum continuous access frequency value under each path, and generates path access segment statistics results. The rate threshold comparison submodule filters the maximum number of consecutive accesses in each path record based on the path access segment statistics, extracts the maximum access frequency value under each path, and compares it with the rate limit threshold item by item to filter the set of path IDs that exceed the threshold and generate abnormal path logs.

4. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The path access control module includes: Based on the abnormal path log, the blocking tag extraction submodule extracts the path identifier field as the key item for retrieval, retrieves the current access control tag of the path, collects the current system time as a unified reference time, and integrates them to generate an access status tag list. The blocking status mapping submodule, based on the access status tag list and according to the default blocking period, compares the NTP time in each path record with the first blocking time of the abnormal path, calculates the remaining blocking duration at the current moment, and if the duration is greater than 0 seconds, retains the current path's blocking status unchanged. At the same time, it constructs a mapping relationship between the path and the remaining blocking duration using the path ID as an index, and generates a path blocking status table. The periodic label update submodule periodically traverses all current entries in the path blocking status table, determines whether the remaining blocking time field of each path is less than or equal to 0 seconds, and if the condition is met, updates the access control label of the corresponding path to allow, records the update event, and generates a compliance status transition audit log.

5. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The task priority sorting module includes: The entry time extraction submodule extracts the creation time field recorded in each task path according to the key task indicators and classifies them by level. It also parses whether there are relay or transfer records in the task field, constructs a task entry detail table, and generates a task entry time sequence. The waiting time calculation submodule calculates the cumulative waiting time of each task from enqueuing to the present based on the task enqueuing time sequence and the current system time, and calculates and outputs the task priority ranking value of each task unit at the current time. The priority queue generation submodule sorts the task sequence in ascending order according to the task priority ranking value. The sorting process is nested according to the task level priority strategy. Within the same level, tasks are arranged according to the ranking value. Between levels, priority dimensions are arranged in order from Level 1 to Level 4 to generate an asset priority queue.

6. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The cloud-based statistical reports include scheduling resource usage, task execution matching degree, and report field structure.

7. The cloud-based automatic statistical report generation system according to claim 1, characterized in that, The cloud-based report generation module includes: The resource status detection submodule obtains the path fields of all tasks to be scheduled in the asset priority queue, and sequentially detects the current schedulable resource indicators of the system, including the number of available CPU cores, available memory capacity and remaining depth of the scheduling queue, summarizes and constructs a resource status parameter set, and generates a resource utilization parameter group. The field mapping calculation submodule performs scheduling adaptation analysis on the correspondence between task path ID and current resource status based on the resource utilization parameter group and the task field in the asset priority queue, analyzes the matching degree between task and current resource status, and establishes field adaptation results. The report structure output submodule sorts the tasks according to the field adaptation results, extracts key fields for all tasks in the matching order, fills them in a structured manner, constructs a standardized output format, and summarizes and integrates them to generate cloud-based statistical reports.

Images