A method, apparatus, electronic device and storage medium for monitoring traffic flow.

Abstract

This application provides a traffic monitoring method, apparatus, electronic device, and storage medium. The method involves: retrieving a first log entry whose timestamp matches the monitoring start time; retrieving a second log entry within the log entry range whose traffic type matches the target traffic type and determining a set of traffic tags; for each traffic tag, initializing the corresponding traffic graph, determining its visual representation within the traffic graph, and filling it in; and outputting the filled traffic graph corresponding to each traffic tag. Since the target network service does not require business process modification or embedding of third-party proxies, visual monitoring of traffic data can be achieved simply by recording traffic logs in a specific format. This decouples the network service from the monitoring system, reducing the intrusion of the monitoring system into core business operations. Furthermore, the monitoring system does not require a separate database to store monitoring data, reducing operational costs. It also does not rely on database components, thus simplifying overall system deployment and reducing resource requirements.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a traffic monitoring method, device, electronic device, and storage medium. Background Technology

[0002] During the operation of network services, a dedicated traffic monitoring system is typically required to intuitively understand their working status and load. Currently, most traffic monitoring systems rely on databases as data storage units, such as the distributed document storage database MongoDB or the relational database management system MySQL, to store and manage the massive amounts of monitoring data obtained from network services. Furthermore, to support the structuring of monitoring data and adapt to database read / write operations, traffic monitoring systems often require custom development of third-party proxies or the introduction of additional data processing components.

[0003] However, these database-based traffic monitoring solutions have revealed several significant shortcomings in actual deployment and application. First, as a complex service component, the database itself relies on numerous technical components for deployment, implementation, and maintenance, resulting in a bloated and complex overall architecture for the monitoring system, sometimes even larger than the core business system it monitors. Second, databases typically have high requirements for system resources such as disk capacity and I / O throughput, increasing both resource overhead and operational pressure. Furthermore, to adapt to monitoring needs, network service applications often require modifications to business processes or the embedding of third-party proxies, creating strong coupling between non-core functions and core services. This not only introduces business intrusion but also increases development and maintenance costs. Summary of the Invention

[0004] In view of this, this application provides a traffic monitoring method, apparatus, electronic device, and storage medium to solve the above-mentioned technical problems.

[0005] In a first aspect of this application, a traffic monitoring method is provided, the method comprising:

[0006] Receive a query request, which includes the monitoring start time, monitoring duration, and target traffic type;

[0007] From the log data stored chronologically by the target network service, retrieve the first log entry whose log timestamp matches the monitoring start time, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value;

[0008] Within the log entry range determined based on the first log entry and the monitoring duration, a second log entry whose traffic type matches the target traffic type is retrieved, and the retrieved second log entry is parsed to obtain a traffic tag set;

[0009] For each traffic tag in the traffic tag set, initialize the traffic graph corresponding to the traffic tag, and for each second log entry, determine the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and fill the traffic graph with the visual representation;

[0010] Output the populated traffic graph corresponding to each traffic label of the target network service.

[0011] According to one embodiment of this application, retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service includes:

[0012] The log file matching the monitoring start time is determined from the log data stored chronologically by the target network service, and the retrieval interval is initialized to the start and end positions of the log file.

[0013] Perform the retrieval steps, which include:

[0014] According to the preset search strategy, determine the search position within the current search interval, and find the first complete log entry after the search position within the search interval; compare the log timestamp of the found log entry with the monitoring start time; if the log timestamp of the log entry is the same as the monitoring start time, then determine the log entry as the first log entry; otherwise, update the search interval according to the comparison result.

[0015] Repeat the search steps until the first log entry is found, or until a complete log entry cannot be found after the determined search position within the current search range.

[0016] According to one embodiment of this application, updating the retrieval interval based on the comparison result includes:

[0017] If the log timestamp of the log entry is greater than the monitoring start time, then the upper limit of the retrieval interval will be updated to the starting position of the log entry.

[0018] If the log timestamp of the log entry is less than the monitoring start time, the lower limit of the retrieval interval will be updated to the starting position of the log entry.

[0019] According to one embodiment of this application, parsing the retrieved second log entries to obtain a traffic tag set includes:

[0020] Each retrieved second log entry is parsed sequentially, and the parsed data is written into the time series database;

[0021] The time-series database includes an index table and a circular linked list. The index table records all unique traffic tags parsed from the second log entries to form the traffic tag set. The circular linked list contains multiple nodes, each corresponding to a second log entry, which records the log timestamp and traffic value sequence parsed from the second log entry. The order of traffic values ​​in the traffic value sequence depends on the order of traffic tags in the index table.

[0022] According to one embodiment of this application, the initialization of the traffic graph corresponding to the traffic label includes:

[0023] Create a traffic bitmap with a preset pixel size corresponding to the traffic label;

[0024] The traffic bitmap is divided into regions based on a predefined region layout to obtain one or more functional areas, wherein the functional areas include data areas for displaying traffic data;

[0025] Determine the pixel width of the smallest unit of the horizontal coordinate and the pixel height of the smallest unit of the vertical coordinate of the data region.

[0026] According to one embodiment of this application, determining the pixel width of the smallest unit of the horizontal coordinate and the pixel height of the smallest unit of the vertical coordinate of the data region includes:

[0027] The pixel width of the minimum horizontal unit of the data area is determined based on the ratio of the pixel width of the data area to the monitoring duration and the predefined traffic sampling period;

[0028] The ratio of the pixel height of the data region to the maximum traffic value associated with the traffic label is used as the pixel height of the smallest unit of the vertical coordinate of the data region.

[0029] According to one embodiment of this application, the traffic graph includes a data area for displaying traffic data; determining the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag includes:

[0030] The filling position of the visualization in the data area is determined based on the difference between the log timestamp in the second log entry and the monitoring start time, as well as the pixel width of the smallest unit of the horizontal coordinate of the data area.

[0031] The pixel width of the smallest unit of the horizontal coordinate of the data area is used as the fill width of the visualization representation in the data area;

[0032] The fill height of the visualization in the data area is determined based on the traffic value associated with the traffic label in the second log entry and the pixel height of the smallest unit of the vertical coordinate of the data area.

[0033] In a second aspect of this application, a traffic monitoring device is provided, the device comprising:

[0034] A receiving unit is used to receive a query request, which includes the monitoring start time, monitoring duration, and target traffic type.

[0035] The first retrieval unit is used to retrieve a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value.

[0036] The second retrieval unit is used to retrieve a second log entry whose traffic type matches the target traffic type within a log entry range determined based on the first log entry and the monitoring duration, and to parse the retrieved second log entry to obtain a traffic tag set.

[0037] Bitmap filling unit is used to initialize a traffic bitmap corresponding to each traffic tag in the traffic tag set, and for each second log entry, determine the visual representation of the second log entry in the traffic bitmap based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and fill the visual representation into the traffic bitmap;

[0038] The output unit is used to output the filled traffic map corresponding to each traffic tag of the target network service.

[0039] In a third aspect of this application, an electronic device is provided, including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the steps of the method proposed in the above embodiments.

[0040] In a fourth aspect of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when executed by a processor, the machine-executable instructions implement the steps of the method proposed in the above embodiments.

[0041] As can be seen from the above technical solutions, the traffic monitoring method provided in this application involves: receiving a query request containing a monitoring start time, a monitoring duration, and a target traffic type; retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service; retrieving a second log entry whose traffic type matches the target traffic type within a log entry interval determined based on the first log entry and the monitoring duration; parsing the retrieved second log entries to obtain a traffic tag set; for each traffic tag in the set, initializing a traffic graph corresponding to that traffic tag; and for each second log entry, determining the visual representation of that second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and filling the traffic graph with the visual representation; and finally outputting the filled traffic graph corresponding to each traffic tag of the target network service. Since the target network service does not require business process transformation or the embedding of third-party proxies, it only needs to record traffic logs in a specific format to complete the visual monitoring of traffic data. This decouples the network service and the monitoring system, reducing the intrusive impact of the monitoring system on core business. In addition, the monitoring system does not need to store monitoring data in a separate database, which can reduce operation and maintenance costs. Moreover, the monitoring system does not depend on database components and does not require the reservation of a dedicated database server. Therefore, the overall system deployment is simple and the operating resource requirements are low.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a traffic monitoring method provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram of an initialization flow graph provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the area layout of a traffic bitmap provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of a process for determining the visual representation of a second log entry in a flow graph, provided by an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of a process for retrieving a first log entry provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of a time-series database provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the overall architecture of a front-end system and a monitoring system provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of the structure of a traffic monitoring device provided in an embodiment of this application;

[0051] Figure 9 This is a schematic diagram of the hardware structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0054] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0055] During the operation of network services, a dedicated traffic monitoring system is typically required to intuitively understand their working status and load. Currently, most traffic monitoring systems rely on databases as data storage units, such as MongoDB or MySQL, to store and manage the massive amounts of monitoring data obtained from network services. Furthermore, to support the structuring of monitoring data and adapt to database read / write operations, traffic monitoring systems often require custom development of third-party proxies or the introduction of additional data processing components.

[0056] However, these database-based traffic monitoring solutions have revealed several significant shortcomings in actual deployment and application. First, as a complex service component, the database itself relies on numerous technical components for deployment, implementation, and maintenance, resulting in a bloated and complex overall architecture for the monitoring system, sometimes even larger than the core business system it monitors. Second, databases typically have high requirements for system resources such as disk capacity and I / O throughput, increasing resource consumption and operational pressure. Furthermore, to adapt to monitoring needs, network service applications often require modifications to business processes or the embedding of third-party proxies, creating strong coupling between non-core functions and core services. This not only introduces business intrusion but also increases development and maintenance costs.

[0057] In view of this, this application discloses a traffic monitoring method to solve the above-mentioned technical problems.

[0058] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a traffic monitoring method provided in an embodiment of this application. It should be noted that this application does not limit the executing entity of the traffic monitoring method. Any network device, such as a server or network switch, when configured (e.g., by installing software, loading firmware, etc.) to execute the traffic monitoring method, functionally constitutes the traffic monitoring system referred to in this application.

[0059] This traffic monitoring method may include the following steps:

[0060] S101: Receive a query request, which includes the monitoring start time, monitoring duration, and target traffic type.

[0061] In some embodiments, when a user needs to perform traffic monitoring queries, they can select or input query parameters through the interactive interface provided by the front-end system.

[0062] The aforementioned front-end system includes, but is not limited to, browsers, web applications, or mobile terminals. It is a client program based on browser / server architecture (B / S architecture) or client / server architecture (C / S architecture), responsible for human-computer interaction and traffic graph display.

[0063] The query parameters that users need to select or input may include: monitoring start time, which refers to the starting point of traffic monitoring. This can be a precise timestamp (such as a Unix timestamp) or a date and time string that conforms to a specific format (such as the international standard for date and time representation defined by the International Organization for Standardization: ISO 8601); monitoring duration, which refers to the length of time that traffic monitoring needs to be carried out continuously from the monitoring start time. This can be a specific time value (such as "3600 seconds", "60 minutes", etc.); and target traffic type, which refers to the type of network device or data stream that needs to be monitored. This is used to specify the source of the traffic monitoring data. For example, the target traffic type can be a switch, server, gateway, or specific network service application.

[0064] The front-end system can respond to the user's selection or input of query parameters by generating a corresponding query request. This query request carries the monitoring start time, monitoring duration, and target traffic type selected or input by the user. The traffic monitoring system receives this query request from the front-end system to proceed with subsequent processing.

[0065] For example, a network administrator wants to check the traffic status of the company's core switch on a given morning. In the browser's management interface, he selects "2025-06-06 10:01:04" in the "Monitoring Start Time" field, "2 hours" in the "Monitoring Duration" dropdown menu, and checks "Tag1" corresponding to the core switch in the "Target Traffic Type" field. He then clicks the "Generate Report" button. The front-end system then encapsulates the selected query parameters into a query request according to a predefined protocol, such as Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS), and sends it to the traffic monitoring system. The traffic monitoring system receives this query request from the front-end system to proceed with subsequent processing.

[0066] It should be noted that this application embodiment does not limit the source of the query request, and its generation method is not limited to the user-triggered scenario described above. The query request can also be automatically generated by other network devices, software systems, or automated processes without manual intervention. For example, a scheduled task can be automatically executed at 1:00 AM every day. This scheduled task will generate a query request containing predefined query parameters: the monitoring start time is set to midnight of the previous day (e.g., "2025-06-06 00:00:00"), the monitoring duration is set to "24 hours", and the target traffic type is "Tag1" corresponding to the core switch, so as to automatically and periodically summarize and monitor the traffic data of the previous day.

[0067] S102: Retrieve a first log entry whose log timestamp matches the monitoring start time from the log data stored chronologically by the target network service, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value.

[0068] Retrieve the first log entry whose log timestamp matches the monitoring start time in the query request from the massive amount of log data generated by the target network service and stored in chronological order.

[0069] In the embodiments of this application, "target network service" refers to services or devices that can actively or passively generate network traffic data, and whose generated traffic data needs to be monitored. For example, the target network service can be an application running in a network environment, i.e., a network service application, such as a web server, database service, message middleware, or video streaming service.

[0070] In some embodiments, to record network traffic logs generated during the operation of the target network service, a traffic logging module can be integrated into the target network service. This module, as a software component, is packaged together with the target network service components during coding and deployed and run together with the target network service. The traffic logging module is responsible for recording the traffic data generated during the operation of the target network service as log entries according to a predefined log format and persistently storing them.

[0071] The pre-defined log format ensures that each log entry in the log data contains standardized information. Specifically, each log entry in the log data contains at least the following three parts: a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value.

[0072] The log timestamp records the precise moment the log entry was generated. Its precision can be set according to requirements, for example, to the millisecond. The log timestamp can also be interpreted as the end time of the current traffic sampling period.

[0073] Traffic type: Used to identify the type of network device or data stream that generates this traffic. For example, the traffic type can be a switch, server, gateway, or specific network service application. For ease of processing, different traffic types can be represented by predefined tags. For example, the traffic type corresponding to "switch" can be denoted as "Tag1", the traffic type corresponding to "server" can be denoted as "Tag2", and so on.

[0074] At least one traffic tag and its associated traffic value: This is the core data part of a log entry, carrying specific traffic statistics. Traffic tags and their associated traffic values ​​can exist in key-value pairs, where: the key is defined as the traffic tag, used to describe the dimension or name of the traffic metric to be monitored; the value is defined as the traffic value associated with that traffic tag, representing the statistics of that traffic metric within a specific time window (i.e., the traffic data sampling period). For example, if monitoring is performed from the network port dimension, a log entry might contain the following traffic tag and its associated traffic value: {PORT1:100, PORT2:200}, indicating that within a specific time window, the traffic value of port 1 is 100, and the traffic value of port 2 is 200. If monitoring is performed from the traffic direction dimension, a log entry might contain the following traffic tag and its associated traffic value: {Received traffic:1500, Sent traffic:1200, Packet loss:5}, and so on.

[0075] In some embodiments, the target network service can statistically analyze the network traffic it processes in real time. Internally, it maintains two sets of accumulator counters: one set records the cumulative traffic data processed at the previous sampling time (denoted as last_log_count), and the other set records the current cumulative traffic data processed (denoted as cur_count). Simultaneously, the target network service has a built-in timer whose trigger period is set to a predefined traffic data sampling period (e.g., 1 second, 5 seconds, or 10 seconds, etc.). Whenever the timer triggers, the target network service performs the following operations:

[0076] For each traffic tag, obtain its corresponding cur_count and last_log_count, and calculate the difference between cur_count and last_log_count. This difference is the traffic value associated with the traffic tag in the current sampling period. Save the obtained cur_count as the new last_log_count to prepare for the next sampling calculation. The traffic log module records the current timestamp, traffic type, and each traffic tag and its associated traffic value obtained in this calculation as a new log entry.

[0077] In the embodiments of this application, the target network service stores log data strictly in chronological order. In some embodiments, for ease of management and retrieval, the log data can be segmented and stored according to the time dimension. For example, a file storage strategy segmented by day can be adopted, that is, all log entries generated by the target network service each day are centrally stored in a separate log file, which can be named by date (for example, the format of the log file is "YYYYMMDD.log"), and all log entries within the log file are strictly arranged in ascending order of timestamp.

[0078] In some embodiments, retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service includes:

[0079] Based on the monitoring start time in the query request, the corresponding date information is parsed out, and a log file matching the date information is retrieved from the log storage path. If no matching log file is found, a message indicating retrieval failure is returned to the front-end system. If a matching log file is found, the monitoring start time is converted into a corresponding timestamp, and the first log entry in the log file whose timestamp is equal to the timestamp corresponding to the monitoring start time is retrieved.

[0080] For example, assuming the monitoring start time in the query request is "2025-06-06 10:01:04", firstly, the corresponding date information "2025-06-06" is parsed, and based on this, a log file named "20250606" is retrieved from the log storage path. Then, the monitoring start time "2025-06-06 10:01:04" is converted to the corresponding timestamp "1749175264", and the log file is searched sequentially (for example, reading log entries one by one from the beginning of the file and comparing their timestamps with the timestamps corresponding to the monitoring start time, or using the characteristic that all log entries within the log file are strictly arranged in ascending order of timestamps to perform a binary search, etc.), to find the first log entry whose timestamp is equal to the timestamp "1749175264" corresponding to the monitoring start time.

[0081] S103: Within the log entry range determined based on the first log entry and the monitoring duration, retrieve a second log entry whose traffic type matches the target traffic type, and parse the retrieved second log entry to obtain a traffic tag set.

[0082] Based on the first log entry retrieved in S102 and the monitoring duration in the query request, a log entry interval to be analyzed is determined. This log entry interval logically corresponds to a complete monitoring time window. The starting point of the log entry interval is the first log entry, and the log timestamp of the first log entry is the start time of the aforementioned monitoring time window. The end point of the log entry interval is determined by adding the log timestamp of the first log entry to the monitoring duration in the query request to obtain the end time of the aforementioned monitoring time window. Then, in the corresponding log file, a retrieval method similar to S102 is used to locate the log entry whose log timestamp is equal to or closest to the end time, and this is taken as the end point of the log entry interval.

[0083] For example, continuing from example S102, if the monitoring start time is 2025-06-06 10:01:04 and the monitoring duration is 2 hours, then the end time of the monitoring time window is 2025-06-06 12:01:04. The system will locate the log entry in the log file 20250606.log whose timestamp is equal to or closest to this end time. All log entries from the first log entry to this log entry constitute the log entry range for this analysis. It should be noted that if the monitoring duration is long, the logically corresponding monitoring time window of the log entry range may span multiple log files segmented by day. This monitoring system has the ability to handle such situations, extracting log entries within the corresponding monitoring time window from multiple consecutive log files and merging them into a complete log entry range.

[0084] After determining the log entry range, a second log entry matching the target traffic type is retrieved within that range. Specifically, each log entry within the log entry range can be traversed, and the log entries can be filtered based on the traffic type recorded in them. Only log entries whose traffic type is the same as the target traffic type in the query request are retained. These filtered log entries constitute the second log entry described in this step.

[0085] All retrieved second log entries are parsed to obtain a set of traffic tags. Specifically, the "at least one traffic tag and its associated traffic value" section of each second log entry can be read, and all traffic tags can be extracted. For example, when traffic tags and their associated traffic values ​​exist in the form of key-value pairs, the key is extracted for each key-value pair; this key is a traffic tag. The traffic tags extracted from all second log entries are then summarized and deduplicated to form the traffic tag set described in this step.

[0086] For example: Suppose the target traffic type in the query request is "Tag1" (corresponding to "switch"). There are 10 log entries within a defined log entry range. Each log entry within this range is iterated through, and filtered based on the traffic type recorded in the log entry. After filtering, 3 entries have the traffic type field "Tag1", and these 3 log entries are the second log entries. All retrieved second log entries are parsed. Assuming the extracted traffic tags are "PORT1, PORT2", "PORT1", and "PORT2, PORT3", the resulting traffic tag set is {PORT1, PORT2, PORT3}.

[0087] S104: For each traffic tag in the traffic tag set, initialize the traffic graph corresponding to the traffic tag, and for each second log entry, determine the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and fill the traffic graph with the visual representation.

[0088] In the embodiments of this application, the filtered, discrete log data (second log entries) are transformed into an intuitive, time-series graphical report, i.e., a traffic graph. This process mainly includes two stages: First, a traffic graph is created and initialized for each traffic tag in the traffic tag set; then, each data point (second log entry) is converted into a visual representation within the traffic graph based on its log timestamp, the traffic value associated with the traffic tag, and the monitoring start time in the query request, and is sequentially filled into the traffic graph.

[0089] The initialization described above refers to creating and configuring a structured chart framework for each traffic tag in the traffic tag set obtained as described in S103. During initialization, some functional areas and a coordinate system for data mapping can be predefined in this framework, but no specific traffic data has been entered at this point.

[0090] like Figure 2 As shown, Figure 2 This is a flowchart illustrating the initialization of a traffic graph according to an embodiment of this application. In some embodiments, initializing the traffic graph corresponding to the traffic tag specifically includes the following steps:

[0091] S201: Create a traffic bitmap with a preset pixel size corresponding to the traffic label.

[0092] Create a bitmap with a preset pixel size for the currently processed traffic label, which will serve as the canvas for drawing the final traffic map.

[0093] For example, the same preset pixel size can be set for all traffic tags, such as 800 pixels wide × 600 pixels high. Alternatively, different preset pixel sizes can be set for different traffic tags according to specific rules, such as the importance of the traffic tag or user-defined configurations.

[0094] S202: Divide the traffic bitmap into regions based on a predefined region layout to obtain one or more functional areas, wherein the functional areas include data areas for displaying traffic data.

[0095] Based on a predefined area layout, the traffic bitmap created above is divided into regions to obtain one or more functional areas with clearly defined functions, thereby forming a structured chart framework. These functional areas include at least a data area for displaying traffic data, and may also include a title area, an axis area, and / or a summary area, etc., but this embodiment does not specifically limit this.

[0096] like Figure 3 As shown, Figure 3 This is a schematic diagram of the area layout of a traffic bitmap provided in an embodiment of this application. The pixel size of the traffic bitmap is 800 pixels wide × 600 pixels high. The functional areas of the traffic bitmap include a data area, a header area, a coordinate axis area, and a summary area. The specific definitions and layouts of each functional area are as follows:

[0097] Data Area: Located in the center of the flow bitmap, it is used to present a curve or bar chart, etc., showing how flow values ​​change over time. The data area has a pixel size of 750 pixels wide × 450 pixels high. In the global coordinate system with the bottom left corner of the flow bitmap as the origin (0,0), the coordinates of the bottom left corner vertex of the data area are (50,100). In the coordinate system of the data area itself, with its bottom left corner as the origin, the horizontal axis represents time, extending horizontally to the right from the origin, with the origin corresponding to the monitoring start time; the vertical axis represents flow value, extending vertically upwards from the origin, with the flow value corresponding to the origin being 0.

[0098] Title area: Located directly above the data area, it displays the title of the traffic graph (such as the traffic label name). The title area is 750 pixels wide × 50 pixels high, and its lower left vertex coordinates are (50, 550) in the global coordinate system mentioned above.

[0099] The coordinate axis area can be further divided into a horizontal axis area and a vertical axis area. The horizontal axis area is located directly below the data area and is used to display the time axis scale and labels. Its pixel dimensions are 750 pixels wide × 50 pixels high, and in the aforementioned global coordinate system, its lower left vertex coordinates are (50, 50). The vertical axis area is located to the left of the data area and is used to display the flow rate scale and labels. Its pixel dimensions are 50 pixels wide × 450 pixels high, and in the aforementioned global coordinate system, its lower left vertex coordinates are (0, 100).

[0100] Summary area: Located directly below the horizontal axis area, it is used to display statistical summary information such as average flow rate and peak flow rate. Its pixel size is 750 pixels wide × 50 pixels high. In the global coordinate system mentioned above, the coordinates of its lower left vertex are (50,0).

[0101] S203: Determine the pixel width of the smallest unit of the horizontal coordinate and the pixel height of the smallest unit of the vertical coordinate of the data area.

[0102] To establish an accurate mapping from data (time, flow value) to pixels within a data area, two key scaling parameters need to be determined: the pixel width of the smallest unit on the horizontal axis and the pixel height of the smallest unit on the vertical axis.

[0103] The minimum unit on the horizontal axis refers to one traffic data sampling period, and the pixel width of the minimum unit on the horizontal axis is defined as the pixel width that each traffic sampling period should occupy within the data area; the minimum unit on the vertical axis refers to a unit traffic value (1 traffic unit), and the pixel height of the minimum unit on the vertical axis is defined as the pixel height that each unit traffic value should occupy within the data area.

[0104] The pixel width of the smallest unit of the horizontal axis and the pixel height of the smallest unit of the vertical axis in the above data area can be predefined and stored constants, which the monitoring system can directly obtain when needed.

[0105] In some embodiments, determining the pixel width of the smallest horizontal coordinate unit and the pixel height of the smallest vertical coordinate unit of the data region specifically includes the following steps:

[0106] S2031: Determine the pixel width of the minimum horizontal unit of the data area based on the ratio of the pixel width of the data area to the monitoring duration and the predefined traffic sampling period.

[0107] Specifically, the pixel width of the smallest unit on the horizontal axis of the data area is calculated using the following formula: Pixel width W = (Pixel width of data area L / Monitoring duration T) × Traffic sampling period N. It should be noted that, to ensure the accuracy of the calculation, the monitoring duration T and the traffic sampling period N in the above formula must use the same time unit, for example, seconds.

[0108] In the coordinate system of the data area itself, the horizontal axis represents time, and the total pixel width L on the horizontal axis corresponds to the entire monitoring duration T. Therefore, the ratio of the pixel width L of the data area to the monitoring duration T represents the pixel width occupied per second. Multiplying this ratio by the traffic sampling period N yields the pixel width that should be allocated on the horizontal axis for each traffic sampling period, which is also the pixel width W of the smallest unit on the horizontal axis.

[0109] For example: Assuming the data area has a pixel width L = 750 pixels, the monitoring duration T = 50 seconds, and the traffic sampling period N = 5 seconds, then the pixel width of the smallest unit on the horizontal axis W = (750 / 50) × 5 = 75 pixels. This means that each 5-second sampling period occupies a width of 75 pixels on the horizontal axis.

[0110] S2032: The ratio of the pixel height of the data area to the maximum flow value associated with the flow label is used as the pixel height of the smallest unit of the vertical coordinate of the data area.

[0111] The pixel height of the smallest unit of the vertical coordinate of the data area is calculated as follows: H = pixel height of the data area G / maximum traffic value Vmax associated with this traffic label.

[0112] In the coordinate system of the data area itself, the vertical axis represents the flow rate value, and the total pixel height of the vertical axis corresponds to the maximum flow rate value Vmax. This maximum flow rate value Vmax refers to the maximum value among all the flow rate values ​​associated with this flow rate tag in all second log entries. The above calculation formula maps the pixel height G of the entire data area to a numerical range from 0 to the maximum flow rate value Vmax, thereby determining the pixel height H corresponding to each unit flow rate value (e.g., 1 Mbps).

[0113] For example: assuming the pixel height of the data area is G = 450 pixels, and the maximum traffic value associated with this traffic tag is Vmax = 150 Mbps, then the pixel height of the smallest unit on the vertical axis is H = 450 / 150 = 3 pixels / Mbps. This means that every 1 Mbps of traffic value corresponds to a height of 3 pixels on the vertical axis.

[0114] After initializing the traffic graph corresponding to this traffic label, the second log entries selected in S103 will be processed one by one. Based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with this traffic label in the second log entry, the abstract traffic data in the second log entry will be converted into a specific visual representation (including but not limited to pixel matrix, rectangular bars, or data points) in the traffic graph data area, and the visual representation will be filled into the traffic graph.

[0115] like Figure 4 As shown, Figure 4 This is a schematic diagram of a process for determining the visual representation of a second log entry in a flow graph, provided by an embodiment of this application.

[0116] In some embodiments, the above visualization is represented as a pixel matrix. Determining the visualization representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag specifically includes the following steps:

[0117] S401: Determine the filling position of the visualization representation in the data area based on the difference between the log timestamp in the second log entry and the monitoring start time, as well as the pixel width of the smallest unit of the horizontal coordinate of the data area.

[0118] In the data area's own coordinate system, with its lower left corner as the origin, the horizontal axis to the right represents increasing time, and the origin corresponds to the monitoring start time. The visualization's fill position in the data area refers to the pixel distance of the lower left corner of the visualization from the origin on the horizontal axis of the data area's own coordinate system.

[0119] Specifically, the visual representation of the filling position P in the data area is calculated as follows: P = [(log timestamp in the second log entry - monitoring start time) / traffic sampling period N] * pixel width W of the smallest unit of the horizontal axis.

[0120] It should be noted that the "log timestamp in the second log entry - monitoring start time" in the above calculation formula is a time difference, which must use the same time unit (e.g., seconds) as the traffic sampling period N. The result of the calculation "time difference / traffic sampling period N" indicates which traffic sampling period this second log entry belongs to in this traffic monitoring, with the sequence number starting from 0. Multiplying this sequence number by the pixel width W of the smallest unit of the horizontal coordinate gives the starting pixel position of the lower left corner vertex of the visualization representation of this second log entry on the horizontal axis of the data area's own coordinate system.

[0121] For example: Given that the monitoring start time is 2025-06-06 10:01:04, the pixel width of the data area is L=750 pixels, the monitoring duration is T=50 seconds, and the traffic sampling period is N=5 seconds, the pixel width of the smallest unit of the horizontal axis is calculated to be W=(750 / 50)×5=75 pixels.

[0122] Example 1 (Second Log Entry A): Its log timestamp is the monitoring start time 2025-06-06 10:01:04. Its corresponding visual representation is located at position P in the data area. A =[(04 seconds - 04 seconds) / 5 seconds] × 75 pixels = 0 pixels. This means that the second log entry A is the log entry of the 0th traffic sampling period in this traffic monitoring. Its corresponding lower left corner vertex in the visualization is on the horizontal axis of the data area's own coordinate system, 0 pixels away from the origin, that is, located at 0 pixels on the horizontal coordinate of the data area.

[0123] Example 2 (Second log entry B): Its log timestamp is 2025-06-06 10:01:19. Its corresponding visual representation is located at position P in the data area. B =[(19 seconds - 04 seconds) / 5 seconds] × 75 pixels = 225 pixels. This means that the second log entry B is the log entry of the third traffic sampling period in this traffic monitoring. Its corresponding lower left corner vertex in the visualization is 225 pixels away from the origin on the horizontal axis of the data area's own coordinate system, that is, it is located at 225 pixels on the horizontal coordinate of the data area.

[0124] S402: Use the pixel width of the smallest unit of the horizontal coordinate of the data area as the fill width of the visualization representation in the data area.

[0125] The fill width of the visualization within the data area refers to the pixel width occupied by the visualization in the horizontal direction. Each second log entry records the traffic value within a complete traffic sampling period; therefore, its corresponding visualization should occupy the width of one traffic sampling period on the horizontal axis. Thus, the fill width X of the visualization within the data area can be directly set to the pixel width W of the smallest unit of the horizontal coordinate of the data area.

[0126] Continuing from the previous example: Regardless of which second log entry it is, its visual representation has a padding width of 75 pixels.

[0127] S403: Determine the fill height of the visualization in the data area based on the traffic value associated with the traffic label in the second log entry and the pixel height of the smallest unit of the vertical coordinate of the data area.

[0128] In the coordinate system of the data region itself, with its lower left corner as the origin, the vertical axis represents increasing flow values, and the flow value corresponding to the origin is 0. The visual representation of the fill height in the data region refers to the pixel height calculated vertically from the baseline (horizontal axis).

[0129] Specifically, the visual representation of the fill height Y in the data area is calculated as follows: Y = Traffic value associated with this traffic label in the second log entry * Pixel height H of the smallest unit on the ordinate. That is, the traffic value associated with this traffic label in the second log entry is proportionally mapped to the pixel height within the data area.

[0130] For example: Given a data area pixel height G = 450 pixels, and a maximum traffic value Vmax associated with this traffic tag = 150 Mbps, the pixel height of the smallest unit on the ordinate is calculated to be H = 450 / 150 = 3 pixels / Mbps. Assuming that in the second log entry B, the traffic value associated with this traffic tag is 100 Mbps, then its corresponding visual representation in the data area would have a fill height Y. B =100Mbps × 3 pixels / Mbps = 300 pixels.

[0131] Based on the combined results, for the second log entry B, a pixel matrix with a width of 75 pixels and a height of 300 pixels will be drawn in the data area of ​​the traffic graph corresponding to this traffic label, with the position 225 pixels away from the origin on the horizontal axis of the data area's own coordinate system as the lower left vertex.

[0132] Once all second log entries have been processed using the steps described above, a complete traffic graph can be generated. This traffic graph visually displays the trend of the traffic value of this traffic tag under the target traffic type changing over time, starting from the monitoring start time and continuing throughout the entire monitoring period.

[0133] S105: Output the filled traffic graph corresponding to each traffic label of the target network service.

[0134] The visualization results generated by the aforementioned steps, namely the traffic graphs corresponding to each traffic label of the target network service and populated with traffic data, are sent to the user or the upper-layer system to complete the presentation of traffic monitoring information.

[0135] In some embodiments, the monitoring system can encode each generated, populated traffic graph into a standard bitmap format, such as Portable Network Graphics (PNG) or Joint Photographic Experts Group (JPEG), forming a binary data stream. This binary data is then directly transmitted to the front-end system, which, upon receiving the data, decodes and renders it in a designated display control (such as an image component or canvas) to present the traffic graph to the user.

[0136] The traffic monitoring method provided in this application involves: receiving a query request containing a monitoring start time, a monitoring duration, and a target traffic type; retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service; retrieving a second log entry whose traffic type matches the target traffic type within a log entry interval determined based on the first log entry and the monitoring duration; parsing the retrieved second log entries to obtain a set of traffic tags; initializing a traffic graph corresponding to each traffic tag for each traffic tag; and for each second log entry, determining a visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and filling the traffic graph with the visual representation; and finally outputting the filled traffic graph corresponding to each traffic tag of the target network service. Since the target network service does not require business process transformation or the embedding of third-party proxies, it only needs to record traffic logs in a specific format to complete the visual monitoring of traffic data. This decouples the network service and the monitoring system, reducing the intrusive impact of the monitoring system on core business. In addition, the monitoring system does not need to store monitoring data in a separate database, which can reduce operation and maintenance costs. Moreover, the monitoring system does not depend on database components and does not require the reservation of a dedicated database server. Therefore, the overall system deployment is simple and the operating resource requirements are low.

[0137] In some embodiments, retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service specifically includes the following steps:

[0138] S1021: Determine the log file that matches the monitoring start time from the log data stored chronologically by the target network service, and initialize the retrieval interval to the start and end positions of the log file.

[0139] The target network service stores log data strictly in chronological order. For ease of management and retrieval, log data can be segmented and stored along a time dimension, such as using a daily file storage strategy. This means all log entries generated by the target network service each day are stored in a single, independent log file, which can be named by date (e.g., "YYYYMMDD.log"). Furthermore, all log entries within this file are strictly arranged in ascending order of timestamps.

[0140] Upon receiving a query request, the system parses out the corresponding date information based on the monitoring start time in the query request, and then retrieves the log file that matches the date information from the log storage path.

[0141] Scenario 1: If no matching log file is found, it indicates that there is no monitoring data at the specified monitoring start time, and the monitoring system can return a retrieval failure message to the front-end system.

[0142] Case 2: If a matching log file is found, obtain the file size of the log file (denoted as N bytes), and initialize the search interval to the entire file range, that is, from the beginning position 0 of the file to the end position N of the file. The initialized search interval is denoted as (0,N), and continue to execute S1022.

[0143] For example: Suppose the monitoring start time in the query request is "2025-06-06 10:01:04", the parsed date information is "2025-06-06", and a log file named "20250606.log" is successfully retrieved in the log storage path. The file size of the log file (N bytes) is obtained, and the initial search range is (0,N), and S1022 is executed.

[0144] S1022: Perform the retrieval step, which includes:

[0145] According to the preset retrieval strategy, determine the retrieval position within the current retrieval interval, and find the first complete log entry after the retrieval position within the retrieval interval; compare the log timestamp of the found log entry with the monitoring start time; if the log timestamp of the log entry is the same as the monitoring start time, then determine the log entry as the first log entry; otherwise, update the retrieval interval according to the comparison result.

[0146] Specifically, a search position is calculated within the current search interval according to a preset search strategy. This preset search strategy can be a bisection method (taking the midpoint of the interval), the golden ratio, or other fixed or dynamic proportions. For ease of understanding, the following explanation uses a bisection method as an example. For instance, if the current search interval is (0, N), the search position determined within the current search interval is at byte N / 2 of the log file.

[0147] Since the calculated search position may be located inside a log entry, making it impossible to effectively obtain the log timestamp, after determining the search position within the current search interval, the first complete log entry following that search position is searched within the current search interval. Specifically, this can be achieved by identifying the separator between log entries (e.g., the newline character "\n") to find the first complete log entry after the search position.

[0148] Parse the first complete log entry after the found search location and extract its log timestamp. To ensure accurate comparison, the log timestamp and the monitoring start time must be converted to the same time unit and format (e.g., uniform to Unix timestamp or formatted seconds).

[0149] If the log timestamp of the log entry is the same as the monitoring start time in the query request, then the log entry is the first log entry to be found, and the retrieval ends successfully.

[0150] If the log entry's timestamp differs from the monitoring start time, the search range needs to be further updated based on the comparison results. Specifically, if the log entry's timestamp is greater than the monitoring start time, it indicates that the first log entry to be searched may be located above this log entry. In this case, the upper limit of the search range is updated to the starting position (starting byte position) of the log entry to narrow the search scope to the range above this log entry. If the log entry's timestamp is less than the monitoring start time, it indicates that the first log entry to be searched may be located below this log entry. In this case, the lower limit of the search range is updated to the starting position (starting byte position) of the log entry to narrow the search scope to the range below this log entry.

[0151] S1023: Repeat the search steps until the first log entry is found, or until a complete log entry cannot be found after the determined search position within the current search range.

[0152] Repeat the retrieval steps described in S1022 until one of the following occurs: a log entry with the same timestamp as the monitoring start time is found, indicating that the log entry is the first log entry to be found, and the retrieval ends successfully; or no complete log entry can be found within the current retrieval interval, indicating that the retrieval has failed.

[0153] like Figure 5 As shown, Figure 5 This is a schematic diagram of a process for retrieving a first log entry provided in an embodiment of this application. Figure 5 The image shows individual log entries in a log file named "20250606.log", all of which are arranged strictly in ascending order of timestamps.

[0154] Assume the log file size is N bytes, the initial search range is (0, N), and the monitoring start time is 2025-06-06 10:02:04.

[0155] First round of retrieval: Calculate the retrieval position within the current retrieval interval (0, N), which is the midpoint position (0+N) / 2. This retrieval position N / 2 falls within the log entry "2025-06-06T10:08:04 Tag4 Device:444444,Port1:89,Port2:32". Within the retrieval interval (0, N), by identifying the newline character "\n" between log entries, find the first complete log entry after the retrieval position N / 2, thus locating the log entry "2025-06-06T10:11:04 Tag9 Device:999999,Port1:3,Port2:32". Since the log timestamp of this log entry "2025-06-06 10:11:04" is greater than the monitoring start time "2025-06-06 10:02:04", the retrieval interval is updated to (0, P), where P is the starting byte position of this log entry.

[0156] The second round of retrieval: Calculate the retrieval position within the current retrieval interval (0, P), which is the midpoint position (0+P) / 2. This retrieval position P / 2 falls within the log entry "2025-06-06T10:02:40 Tag2 Device:abcdef,Port1:33,Port2:33". Within the retrieval interval (0, P), by identifying the newline character "\n" between log entries, find the first complete log entry after the retrieval position P / 2, thus locating the log entry "2025-06-06T10:02:46 Tag1 Device:123456,Port1:176". Since the log timestamp of this log entry "2025-06-06 10:02:46" is greater than the monitoring start time "2025-06-06 10:02:04", the retrieval interval is updated to (0, Q), where Q is the starting byte position of this log entry.

[0157] The third round of retrieval: Calculate the retrieval position within the current retrieval interval (0, Q), which is the midpoint position (0+Q) / 2. This retrieval position Q / 2 falls within the log entry "2025-06-06T10:02:04 Tag9 Device:999999,Port1:54,Port2:32". Within the retrieval interval (0, Q), by identifying the newline character "\n" between log entries, find the first complete log entry after the retrieval position Q / 2, thus locating the log entry "2025-06-06T10:02:04 Tag10 Device:aaaaaa,Port1:77,Port2:32". Since the log timestamp of this log entry "2025-06-06 10:02:04" is equal to the monitoring start time "2025-06-06 10:02:04", this log entry is the first log entry to be found, and the retrieval ends successfully.

[0158] In some embodiments, parsing the retrieved second log entries to obtain a traffic tag set specifically includes the following steps:

[0159] Each retrieved second log entry is parsed sequentially, and the parsed data is written into a time-series database. The time-series database contains an index table and a circular linked list. The index table records all unique traffic tags parsed from the second log entries to form the traffic tag set. The circular linked list contains multiple nodes, each corresponding to a second log entry, which records the record timestamp and traffic value sequence parsed from the second log entry. The order of traffic values ​​in the traffic value sequence depends on the order of traffic tags in the index table.

[0160] Specifically, a time-series database corresponding to the target traffic type can be created. This time-series database is a software module residing in computer memory, consisting of an index table (denoted as DL) and a circular linked list (denoted as DQ). Therefore, it does not rely on traditional database components, making deployment easier and system maintenance simpler. In addition, the monitoring system does not need to store large amounts of index data and serialized monitoring data, resulting in less reliance on disk resources and shorter storage links, thus avoiding the risk of data loss due to transfer from the source.

[0161] In this embodiment, each second log entry is traversed sequentially, parsed, and all traffic tags contained therein are extracted.

[0162] The aforementioned index table DL is used to record and maintain all unique traffic tags parsed from the second log entries. The collection of these traffic tags constitutes the final traffic tag set. Specifically, when parsing the second log entries, each newly appearing traffic tag can be dynamically added to the index table DL, while a traffic tag that already exists in the index table DL will not be added again.

[0163] For example: Suppose there are three second log entries, namely second log entry C: 2025-06-06T10:02:40

[0164] Tag1 Device: 123456, Port1: 48, Port2: 22; Second log entry D: 2025-06-06T10:02:46 Tag1 Device:

[0165] 123456, Port1:176; Second log entry E: 2025-06-06T10:11:04 Tag1 Device:123456, Port1:3,

[0166] Port2:32.

[0167] Each second log entry is traversed sequentially and parsed. First, second log entry C is parsed to obtain traffic labels Port1 and Port2, which are then recorded in the index table DL. Next, second log entry D is parsed to obtain traffic label Port1; since this traffic label already exists in index table DL, it is not recorded again. Finally, second log entry E is parsed to obtain traffic labels Port3 and Port2; Port3 is recorded in index table DL, while Port2, since it already exists, is not recorded again. Ultimately, the set of traffic labels recorded in index table DL is {Port1, Port2, Port3}.

[0168] While iterating through each second log entry, parsing it, and extracting all the traffic tags contained therein, the log timestamps and the traffic values ​​associated with each traffic tag are also extracted simultaneously.

[0169] The circular linked list DQ described above contains multiple nodes, each corresponding to a second log entry. Each node records at least two parts of data: one is the log timestamp parsed from the second log entry; the other is a sequence of flow values, in which the order of the flow values ​​strictly follows the order of the flow tags in the index table.

[0170] For the aforementioned flow value sequence, if a certain second log entry does not contain a flow tag recorded in the index table DL, then the position of that flow tag in the flow value sequence corresponding to its node will be filled with 0, thereby ensuring that the length of the flow value sequence of all nodes is completely consistent with the number of flow tags recorded in the index table DL, which facilitates subsequent unified calculation and retrieval.

[0171] The nodes in the circular linked list are strictly arranged in ascending order according to the log timestamps. The timestamps of adjacent nodes do not have to be consecutive. The data is stored in a skip-style manner, which reduces the unnecessary storage overhead of invalid traffic data. At the same time, the linked list is more efficient when there are fewer nodes.

[0172] Continuing the previous example: the traffic tags recorded in the index table DL are in a fixed order of Port1, Port2, Port3. Each second log entry is traversed sequentially, parsed, and its log timestamp and the traffic value associated with each traffic tag are extracted. First, second log entry C is parsed to obtain the log timestamp "1749175360" and the traffic value sequence {48,22,0}. Since this second log entry does not contain the traffic tag Port3 recorded in the index table DL, the positions corresponding to traffic tag Port3 in the traffic value sequence are filled with 0s. Next, second log entry D is parsed to obtain the log timestamp "1749175366" and the traffic value sequence {176,0,0}. Finally, second log entry E is parsed to obtain the log timestamp "1749175864" and the traffic value sequence {0,32,3}.

[0173] like Figure 6 As shown, Figure 6 This is a schematic diagram of a time-series database provided in an embodiment of this application. The time-series database is a software module residing in computer memory, logically composed of an index table DL and a circular linked list DQ.

[0174] The index table DL is used to uniformly manage all occurrences of unique traffic tags. This index table is accessed through a Node*DL entry pointer, which points to the beginning of the index table DL. Each index entry in the index table DL records a unique traffic tag and contains a Node*next pointer to point to the next index entry. In this way, all traffic tags in the index table DL are chained together, forming a clear order, facilitating traversal during traffic graph initialization. All unique traffic tags together constitute the traffic tag set to be obtained in this application.

[0175] A circular linked list (DQ) is used to record specific traffic values ​​in chronological order. This circular linked list is accessed through a Node*DQ entry pointer, which points to the first node in the list. Each node in the circular linked list is a Node structure, corresponding to a second log entry. Each node contains the following data: Node*pre, a pointer to the previous node; Node*next, a pointer to the next node. Through the Node*pre and Node*next pointers, all nodes are connected into a doubly linked circular list, which preserves the time sequence and supports efficient traversal; Log timestamp, recording the log timestamp in the second log entry, continuing the previous example, such as 1749175360, 1749175366, 1749175864; Flow value sequence, a set of flow values ​​related to the second log entry. The order of each flow value in the flow value sequence strictly follows the order of the flow labels in the index table DL. Continuing the previous example, the order of the flow labels in the index table is Port1, Port2, Port3. Then the flow value sequence of the node corresponding to the second log entry C is {48, 22, 0}, indicating that the flow value of Port1 is 48, the flow value of Port2 is 22, and the flow value of Port3 is not recorded in this log entry, so it is filled with 0.

[0176] In this embodiment, the efficient organization of traffic tags and traffic data is achieved through two modules in the time-series database: the index table DL and the circular linked list DQ.

[0177] In some embodiments, the functionality of the index table DL can be further extended. Specifically, in addition to recording the traffic label, each index entry can also be associated with the traffic peak values ​​recorded for that traffic label across all second log entries, including the maximum traffic value Vmax and the minimum traffic value Vmin.

[0178] Specifically, the initial values ​​of Vmax and Vmin can be set to 0. Before traversing the second log entry and preparing to write a certain flow value to a node in the circular linked list DQ, the Vmax and Vmin corresponding to the flow tag of this flow value in the index table DL can be obtained. This flow value is compared with the current Vmax. If this flow value is greater than the current Vmax, then Vmax is updated to this flow value. At the same time, this flow value is compared with the current Vmin. If this flow value is less than the current Vmin, then Vmin is updated to this flow value.

[0179] like Figure 7 As shown, Figure 7 This is a schematic diagram of the overall architecture of a front-end system and a monitoring system provided in an embodiment of this application.

[0180] Front-end systems include, but are not limited to, browsers, web applications, or mobile terminals. They are client programs based on B / S or C / S architectures, responsible for human-computer interaction and traffic graph display.

[0181] A target network service refers to a service or device that can actively or passively generate network traffic data, and whose generated traffic data needs to be monitored. For example, a target network service can be an application running in a network environment, i.e., a network service application. During operation, the target network service records network traffic data flowing through the application component modules. The network traffic data is recorded once per second and saved through a traffic log module.

[0182] The log management service is a log service component that categorizes log entries by day. All log entries generated by the target network service each day are centrally stored in a separate log file named by date (e.g., the log file format is "20250606.log"), and all log entries within the log file are strictly arranged in ascending order of timestamp.

[0183] Any network device, such as a server or network switch, when configured to execute the traffic monitoring method provided in the embodiments of this application, functionally constitutes the monitoring system referred to in this application.

[0184] The log scanning module is deployed on the monitoring system. Its core functions include: retrieving the first log entry whose log timestamp matches the monitoring start time in the query request from the log data stored chronologically by the target network service; retrieving the second log entry whose traffic type matches the target traffic type within the log entry interval determined based on the first log entry and the monitoring duration; parsing each retrieved second log entry in sequence and writing the parsed data into the real-time database.

[0185] The real-time database is deployed on the monitoring system to implement a simplified time-series database, consisting of an index table (DL) and a circular linked list (DQ). The index table (DL) records and maintains all unique traffic tags parsed from second log entries. The collection of these traffic tags constitutes the final traffic tag set, such as {Port1, Port2, Port3}. The circular linked list (DQ) records specific traffic values ​​in chronological order. The list contains multiple nodes, each corresponding to a second log entry. Each node records the log timestamp parsed from that second log entry and a sequence of traffic values. The order of the traffic values ​​in the sequence strictly follows the order of the traffic tags in the index table (DL). For example, if the traffic tags in the index table (DL) are in the order of Port1, Port2, Port3, then the traffic value sequence for a node corresponding to a certain second log entry would be {48, 22, 0}, indicating that the traffic value for Port1 is 48, the traffic value for Port2 is 22, and the traffic value for Port3 is not recorded in this log entry, so it is filled with 0.

[0186] The traffic graph drawing module is deployed on the monitoring system. Its core functions include: for each traffic tag in the traffic tag set, initializing the traffic graph corresponding to the traffic tag; for each second log entry, determining the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and filling the visual representation into the traffic graph; and outputting the filled traffic graph corresponding to each traffic tag of the target network service.

[0187] Specifically, during operation, the target network service records network traffic data processed by the application component modules. Traffic data is generated every second (i.e., the traffic sampling period is 1 second) and recorded in a log file via the traffic log module. Specifically, the traffic statistics table in the target network service records the `cur_count` and `last_log_count` for traffic label Port1, and the `cur_count` and `last_log_count` for traffic label Port2. `cur_count` represents the current traffic value flowing through Port1 and Port2, and `last_log_count` represents the traffic value flowing through Port1 and Port2 at the end of the previous second. The target network service has a 1-second timer. At the end of each second, the timer task calculates the difference between `cur_count` and `last_log_count` for Port1 and Port2 respectively, recording this difference as the traffic value for Port1 and Port2 within the current sampling period.

[0188] The target network service calls the traffic logging module to record the traffic values ​​generated by Port1 and Port2. For example, the content of log entry L1 is: "2025-06-06T10:02:40 Tag1 Device:123456,Port1:48,Port2:32"; the log timestamp in L1 is 2025-06-06T10:02:40, the traffic type is Tag1, the traffic volume of Port1 is 48, and the traffic volume of Port2 is 32. The traffic logging module sends the log data to the log management service, which writes the log entry to a log file named "20250606.log" and persists it to disk.

[0189] When users need to perform traffic monitoring queries, they can select or input query parameters through the interactive interface provided by the front-end system. The query parameters that users need to select or input include at least the monitoring start time, monitoring duration, and target traffic type. In response to the user's selection or input of query parameters, the front-end system generates a corresponding query request. This query request carries the user's selected or input monitoring start time (2025-06-06 10:02:40, corresponding timestamp 1749175360), monitoring duration (30 seconds), and target traffic type (Tag1). The monitoring system receives this query request sent by the front-end system to proceed with subsequent processing.

[0190] After receiving a query request from the front-end system, the monitoring service uses the log scanning module to parse out the corresponding date information (20250606) based on the monitoring start time in the query request, and then retrieves the log file that matches the date information in the log storage path.

[0191] If no matching log file is found, it indicates that there is no monitoring data at the specified monitoring start time, and the monitoring system can return a retrieval failure message to the front-end system; if a matching log file is found, the first log entry with the same log timestamp as the monitoring start time in the query request is located from the log file in the manner described in S1021 to S1023.

[0192] In this embodiment, the log file 20250606.log can be retrieved in the log storage path, and the log entry L1 that meets the time condition can be located in the log file: 2025-06-06T10:02:40 Tag1 Device:123456,Port1:48,Port2:32.

[0193] Based on the log timestamp of log entry L1 and the monitoring duration in the query request, a log entry range to be analyzed is determined. This log entry range logically corresponds to a complete monitoring time window, namely [2025-06-06 10:02:40, 2025-06-06 10:03:10].

[0194] Within the defined log entry range, a second log entry matching the target traffic type (Tag1) is retrieved. Specifically, each log entry within the log entry range can be traversed, and the log entries can be filtered based on the traffic type recorded in them. Only log entries whose traffic type is the same as the target traffic type (Tag1) in the query request are retained. These filtered log entries constitute the second log entry described in this step.

[0195] In this embodiment, the second log entry includes at least log entry L1: 2025-06-06T10:02:40 Tag1Device:123456,Port1:48,Port2:32; and log entry L2: 2025-06-06T10:02:45.178 Tag1Device:123456,Port1:30.

[0196] Create a real-time database corresponding to the target traffic type. The real-time database is deployed on the monitoring system to implement a simplified time-series database, consisting of an index table DL and a circular linked list DQ.

[0197] Each second log entry is traversed sequentially, parsed, and all traffic tags contained therein are extracted. The index table DL records all unique traffic tags parsed from the second log entries. The collection of these traffic tags constitutes the final traffic tag set. In this implementation, the index table DL has a length of 2, and the record values ​​are Port1 and Port2.

[0198] While sequentially traversing each second log entry, parsing it, and extracting all the traffic tags contained therein, the log timestamps and the traffic values ​​associated with each traffic tag are also extracted simultaneously. The circular linked list DQ contains multiple nodes, each corresponding to a second log entry. Each node records the log timestamp parsed from that second log entry, and a sequence of traffic values. The order of the traffic values ​​in the sequence strictly follows the order of the traffic tags in the index table. In this implementation, each node in the circular linked list DQ contains three elements: a timestamp + two traffic values, corresponding to the log timestamp, Port1 traffic value, and Port2 traffic value, respectively. The length of the circular linked list DL is 30, equal to the monitoring duration. For the node corresponding to log entry L1, the node element values ​​are: 1749175360, 48, and 32; for the node corresponding to log entry L2, the node element values ​​are: 1749175365, 30, and 0.

[0199] The flow graph drawing module queries the index table DL in the real-time database, obtains the flow labels Port1 and Port2 recorded therein, and then creates flow bitmaps Port1.bmp and Port2.bmp with a pixel size of 800*600 for Port1 and Port2 respectively.

[0200] Taking the traffic bitmap Port1.bmp as an example, it is divided into multiple functional areas based on a predefined area layout. These functional areas include a data area for displaying traffic data, a title area for displaying the traffic graph title, coordinate axis areas (horizontal and vertical axis areas), and a summary area for displaying statistical information. The specific area layout can be as follows: Figure 3 As shown.

[0201] As described in S2031 to S2032, the pixel width W of the smallest horizontal unit and the pixel height H of the smallest vertical unit of the data area are determined. In this embodiment, the pixel width L of the data area is 750 pixels, the pixel height G of the data area is 450 pixels, the monitoring duration T is 30 seconds, the traffic sampling period N is 1 second, and the maximum value Vmax of the traffic value associated with Port1 within the range of the second log entry is 150Mbps. Therefore, the pixel width W of the smallest horizontal unit is (750 / 30) × 1 = 25 pixels, and the pixel height H of the smallest vertical unit is 450 / 150 = 3 pixels / Mbps.

[0202] The flow graph drawing module queries the circular linked list in the real-time database. Starting from the node pointed to by Node*DQ, it scans the node data in the circular linked list DQ and obtains the node data 1749175360|48|32, 1749175365|30|0, ... in sequence.

[0203] Node data is processed one by one according to the methods described in S401 to S403. In this embodiment, for log entry L1, a pixel matrix with a width of 25 pixels and a height of 144 pixels is drawn in the data area of ​​the traffic bitmap Port1.bmp, with the lower left vertex located at a position 0 pixels away from the origin on the horizontal axis of the data area's own coordinate system; for log entry L1, a pixel matrix with a width of 25 pixels and a height of 90 pixels is drawn in the data area of ​​the traffic bitmap Port1.bmp, with the lower left vertex located at a position 125 pixels away from the origin on the horizontal axis of the data area's own coordinate system. This process continues until all node data is filled with pixels in the data area of ​​the traffic bitmap Port1.bmp, thus completing the drawing of the Port1.png traffic map.

[0204] Similarly, the traffic graph in Port2.png can also be drawn in the manner described in the above embodiments.

[0205] The monitoring system transmits the completed Port1.png and Port2.png traffic graphs to the front-end system as binary data and deletes the index table DL and the circular linked list DQ from the real-time database. Upon receiving the data, the front-end system decodes and renders it in the designated display control to present the Port1.png and Port2.png traffic graphs to the user.

[0206] The above description describes the method provided in this application. The following description describes the apparatus provided in this application:

[0207] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a traffic monitoring device provided in an embodiment of this application.

[0208] like Figure 8 As shown, the device may include:

[0209] The receiving unit 810 is used to receive a query request, which includes the monitoring start time, monitoring duration and target traffic type.

[0210] The first retrieval unit 820 is used to retrieve a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value.

[0211] The second retrieval unit 830 is used to retrieve a second log entry whose traffic type matches the target traffic type within a log entry interval determined based on the first log entry and the monitoring duration, and to parse the retrieved second log entry to obtain a traffic tag set.

[0212] Bitmap filling unit 840 is used to initialize a traffic bitmap corresponding to each traffic tag in the traffic tag set, and for each second log entry, based on the monitoring start time, the log timestamp in the second log entry and the traffic value associated with the traffic tag, determine the visual representation of the second log entry in the traffic bitmap, and fill the visual representation into the traffic bitmap.

[0213] Output unit 850 is used to output the filled traffic map corresponding to each traffic tag of the target network service.

[0214] Optionally, the first retrieval unit 820 is specifically used for:

[0215] The log file matching the monitoring start time is determined from the log data stored chronologically by the target network service, and the retrieval interval is initialized to the start and end positions of the log file.

[0216] Perform the retrieval steps, which include:

[0217] According to the preset search strategy, determine the search position within the current search interval, and find the first complete log entry after the search position within the search interval; compare the log timestamp of the found log entry with the monitoring start time; if the log timestamp of the log entry is the same as the monitoring start time, then determine the log entry as the first log entry; otherwise, update the search interval according to the comparison result.

[0218] Repeat the search steps until the first log entry is found, or until a complete log entry cannot be found after the determined search position within the current search range.

[0219] Optionally, the first retrieval unit 820 is specifically used for:

[0220] If the log timestamp of the log entry is greater than the monitoring start time, then the upper limit of the retrieval interval will be updated to the starting position of the log entry.

[0221] If the log timestamp of the log entry is less than the monitoring start time, the lower limit of the retrieval interval will be updated to the starting position of the log entry.

[0222] Optionally, the second retrieval unit 830 is specifically used for:

[0223] Each retrieved second log entry is parsed sequentially, and the parsed data is written into the time series database;

[0224] The time-series database includes an index table and a circular linked list. The index table records all unique traffic tags parsed from the second log entries to form the traffic tag set. The circular linked list contains multiple nodes, each corresponding to a second log entry, which records the log timestamp and traffic value sequence parsed from the second log entry. The order of traffic values ​​in the traffic value sequence depends on the order of traffic tags in the index table.

[0225] Optionally, the bitmap filling unit 840 is specifically used for:

[0226] Create a traffic bitmap with a preset pixel size corresponding to the traffic label;

[0227] The traffic bitmap is divided into regions based on a predefined region layout to obtain one or more functional areas, wherein the functional areas include data areas for displaying traffic data;

[0228] Determine the pixel width of the smallest unit of the horizontal coordinate and the pixel height of the smallest unit of the vertical coordinate of the data region.

[0229] Optionally, the bitmap filling unit 840 is specifically used for:

[0230] The pixel width of the minimum horizontal unit of the data area is determined based on the ratio of the pixel width of the data area to the monitoring duration and the predefined traffic sampling period;

[0231] The ratio of the pixel height of the data region to the maximum traffic value associated with the traffic label is used as the pixel height of the smallest unit of the vertical coordinate of the data region.

[0232] Optionally, the bitmap filling unit 840 is specifically used for:

[0233] The filling position of the visualization in the data area is determined based on the difference between the log timestamp in the second log entry and the monitoring start time, as well as the pixel width of the smallest unit of the horizontal coordinate of the data area.

[0234] The pixel width of the smallest unit of the horizontal coordinate of the data area is used as the fill width of the visualization representation in the data area;

[0235] The fill height of the visualization in the data area is determined based on the traffic value associated with the traffic label in the second log entry and the pixel height of the smallest unit of the vertical coordinate of the data area.

[0236] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0237] This application also provides a hardware structure. See [link to relevant documentation]. Figure 9 , Figure 9 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 9 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0238] Based on the same application concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0239] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0240] It should be noted that, in this document, relational terms such as "objective" and "target" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0241] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A traffic monitoring method, characterized in that, The method includes: Receive a query request, which includes the monitoring start time, monitoring duration, and target traffic type; From the log data stored chronologically by the target network service, retrieve the first log entry whose log timestamp matches the monitoring start time, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value; Within the log entry range determined based on the first log entry and the monitoring duration, a second log entry whose traffic type matches the target traffic type is retrieved, and the retrieved second log entry is parsed to obtain a traffic tag set; For each traffic tag in the traffic tag set, initialize the traffic graph corresponding to the traffic tag, and for each second log entry, determine the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and fill the traffic graph with the visual representation; Output the populated traffic graph corresponding to each traffic label of the target network service; The process of parsing the retrieved second log entries yields a set of traffic tags, including: Create a time-series database in memory corresponding to the target traffic type; Each retrieved second log entry is parsed sequentially, and the parsed data is written into the time series database; The time-series database includes an index table and a circular linked list. The index table records all unique traffic tags parsed from the second log entries to form the traffic tag set. The circular linked list contains multiple nodes, each corresponding to a second log entry, which records the log timestamp and traffic value sequence parsed from the second log entry. The order of traffic values ​​in the traffic value sequence depends on the order of traffic tags in the index table.

2. The method according to claim 1, characterized in that, The step of retrieving a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service includes: The log file matching the monitoring start time is determined from the log data stored chronologically by the target network service, and the retrieval interval is initialized to the start and end positions of the log file. Perform the retrieval steps, which include: According to the preset search strategy, determine the search position within the current search interval, and find the first complete log entry after the search position within the search interval; compare the log timestamp of the found log entry with the monitoring start time; if the log timestamp of the log entry is the same as the monitoring start time, then determine the log entry as the first log entry; otherwise, update the search interval according to the comparison result. Repeat the search steps until the first log entry is found, or until a complete log entry cannot be found after the determined search position within the current search range.

3. The method according to claim 2, characterized in that, Updating the retrieval interval based on the comparison results includes: If the log timestamp of the log entry is greater than the monitoring start time, then the upper limit of the retrieval interval will be updated to the starting position of the log entry. If the log timestamp of the log entry is less than the monitoring start time, the lower limit of the retrieval interval will be updated to the starting position of the log entry.

4. The method according to claim 1, characterized in that, The initialization of the traffic graph corresponding to the traffic label includes: Create a traffic bitmap with a preset pixel size corresponding to the traffic label; The traffic bitmap is divided into regions based on a predefined region layout to obtain one or more functional areas, wherein the functional areas include data areas for displaying traffic data; Determine the pixel width of the smallest unit of the horizontal coordinate and the pixel height of the smallest unit of the vertical coordinate of the data region.

5. The method according to claim 4, characterized in that, Determining the pixel width of the smallest horizontal coordinate unit and the pixel height of the smallest vertical coordinate unit of the data region includes: The pixel width of the minimum horizontal unit of the data area is determined based on the ratio of the pixel width of the data area to the monitoring duration and the predefined traffic sampling period; The ratio of the pixel height of the data region to the maximum traffic value associated with the traffic label is used as the pixel height of the smallest unit of the vertical coordinate of the data region.

6. The method according to claim 1, characterized in that, The traffic graph includes a data area for displaying traffic data; determining the visual representation of the second log entry in the traffic graph based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag includes: The filling position of the visualization in the data area is determined based on the difference between the log timestamp in the second log entry and the monitoring start time, as well as the pixel width of the smallest unit of the horizontal coordinate of the data area. The pixel width of the smallest unit of the horizontal coordinate of the data area is used as the fill width of the visualization representation in the data area; The fill height of the visualization in the data area is determined based on the traffic value associated with the traffic label in the second log entry and the pixel height of the smallest unit of the vertical coordinate of the data area.

7. A flow monitoring device, characterized in that, The device includes: A receiving unit is used to receive a query request, which includes the monitoring start time, monitoring duration, and target traffic type. The first retrieval unit is used to retrieve a first log entry whose log timestamp matches the monitoring start time from log data stored chronologically by the target network service, wherein each log entry in the log data contains a log timestamp, a traffic type, and at least one traffic tag and its associated traffic value. The second retrieval unit is used to retrieve a second log entry whose traffic type matches the target traffic type within a log entry range determined based on the first log entry and the monitoring duration, and to parse the retrieved second log entry to obtain a traffic tag set. The process of parsing the retrieved second log entries yields a set of traffic tags, including: Create a time-series database in memory corresponding to the target traffic type; Each retrieved second log entry is parsed sequentially, and the parsed data is written into the time series database; The time-series database includes an index table and a circular linked list. The index table records all unique traffic tags parsed from the second log entries to form the traffic tag set. The circular linked list contains multiple nodes, each corresponding to a second log entry, which records the log timestamp and traffic value sequence parsed from the second log entry. The order of traffic values ​​in the traffic value sequence depends on the order of traffic tags in the index table. Bitmap filling unit is used to initialize a traffic bitmap corresponding to each traffic tag in the traffic tag set, and for each second log entry, determine the visual representation of the second log entry in the traffic bitmap based on the monitoring start time, the log timestamp in the second log entry, and the traffic value associated with the traffic tag, and fill the visual representation into the traffic bitmap; The output unit is used to output the filled traffic map corresponding to each traffic tag of the target network service.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method as described in any one of claims 1-6.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-6.

Images