A data real-time statistics method and system

Abstract

This invention relates to the field of real-time data statistics technology, and discloses a real-time data statistics method and system. The method includes: calculating the total memory space size based on the number of ports on a switch; requesting a shared memory region using a system unique key value and mapping the shared memory region to the process address space; creating a timer object; triggering a callback function when the timer object's period expires and writing the read statistical data into the shared memory region; and when the front-end application module receives a statistical data retrieval request, calling an access interface function to directly read the port statistical data from the corresponding memory address in the shared memory region. This invention, by establishing a shared memory-based data buffering mechanism combined with a timer-driven background periodic collection strategy, decouples the collection and access of port statistical data, making it suitable for network monitoring scenarios requiring high-frequency refresh of statistical information and effectively improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of real-time data statistics technology, and in particular to a real-time data statistics method and system. Background Technology

[0002] In network switch systems, port statistics include the number of bytes sent, bytes received, packets sent, packets received, dropped packets, and erroneous packets. When users need to query port statistics through command-line interfaces, web management interfaces, or internal monitoring processes, they directly access the hardware registers of the switch chip synchronously via the MDIO bus or I2C bus for real-time reading. However, in multi-port switch scenarios, this direct access to chip registers requires accessing the hardware registers port by port and statistical item by item via the bus. This process is a synchronous blocking operation, with a single complete read taking hundreds of milliseconds or even seconds. When multiple processes concurrently request statistics, frequent bus access and chip register read operations severely consume CPU resources, causing bus resource contention and a sharp decline in overall system performance. Users experience noticeable lag when querying, making it impossible to meet the needs of high-frequency, real-time monitoring. Summary of the Invention

[0003] This invention provides a real-time data statistics method and system. By establishing a data buffer mechanism based on shared memory and combining it with a timer-driven background periodic collection strategy, this invention decouples the collection and access of port statistics data. It is suitable for network monitoring scenarios that require high-frequency refresh of statistical information and effectively improves the user experience.

[0004] In a first aspect, the present invention provides a real-time data statistics method, the real-time data statistics method comprising:

[0005] The total memory space size is calculated based on the number of ports on the switch, a shared memory region is requested using a system-unique key value, and the shared memory region is mapped to the process address space.

[0006] A timer object is created. When the timer object's period expires, a callback function is triggered and the read statistical data is written to the shared memory area.

[0007] When the front-end application module receives a request to obtain statistical data, it calls the access interface function to directly read the port statistical data from the memory address corresponding to the shared memory region.

[0008] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, the total memory space size is calculated based on the number of ports on the switch, a shared memory region is requested using a system unique key, and the shared memory region is mapped to the process address space, including:

[0009] Read the physical port configuration information of the switch to obtain the number of ports. Determine the number of statistical items for each port, the data type and storage bytes of each statistical item according to the preset statistical item type list, and calculate the total memory space required to store all statistical items for all ports.

[0010] Generate a system-unique key value, and based on the total memory space size and the system-unique key value, apply for a shared memory region and generate a shared memory identifier;

[0011] Based on the shared memory identifier, the shared memory region is mapped to the process address space, the port data blocks are divided according to the port number order, and the offset position is set according to the statistical item type.

[0012] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, mapping the shared memory region to the process address space based on the shared memory identifier, dividing port data blocks according to port number order and setting offset positions according to statistical item types includes:

[0013] Based on the shared memory identifier, the shared memory attach interface function is called to map the shared memory region to the process address space, obtain the starting address pointer, and divide the shared memory region into port data blocks according to the port number order;

[0014] Within each port data block, the offset positions of each statistical item are set in a fixed order according to the statistical item type, and a zeroing initialization operation is performed on the shared memory region.

[0015] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, setting the offset positions of each statistical item in a fixed order according to the statistical item type within each port data block and performing a zeroing initialization operation on the shared memory region includes:

[0016] Within each port data block, offset positions for the statistical items of number of bytes sent, number of bytes received, number of packets sent, and number of packets received are assigned in the order of statistical item type.

[0017] Set all bytes in the shared memory region to zero, so that the initial data of each statistical item is zero.

[0018] In conjunction with the first aspect, in the fourth implementation of the first aspect of the present invention, a timer object is created, and when the period of the timer object expires, a callback function is triggered and the read statistical data is written into the shared memory area, including:

[0019] Create a timer object and configure the periodic trigger time, and define a callback function containing the data collection and update logic;

[0020] Register the callback function as the expiration handling function of the timer object, configure the timer object to periodic trigger mode, and call the startup interface to activate the timer object;

[0021] When the timer object expires, a callback function is triggered to read the statistical data of all ports in the switching chip in batches through the bus interface, and write the read statistical data into the shared memory area according to the port number and offset position.

[0022] In conjunction with the first aspect, in the fifth implementation of the first aspect of the present invention, when the timer object's period expires, a callback function is triggered to read statistical data from all ports in the switching chip in batches through the bus interface, and the read statistical data is written into the shared memory region according to the port number and offset position, including:

[0023] When the timer object's period expires, a callback function is triggered, which calls the underlying driver interface to read all port statistics of the switching chip in batches through the bus, and organizes the read statistics into a temporary data buffer.

[0024] The statistical data read from the temporary data buffer is traversed, the target address of the shared memory region is calculated based on the port number and offset position, and the read statistical data is written into the shared memory region based on the target address.

[0025] In conjunction with the first aspect, in a sixth implementation of the first aspect of the present invention, after writing the read statistical data into the shared memory region, the method further includes:

[0026] Define an access interface function and set the port number and statistical item type as input parameters, and validate the validity of the input parameters;

[0027] Based on the system's unique key value, the shared memory attachment interface is called to map the shared memory region to the process address space, and the memory address of the target statistical item in the shared memory region is calculated according to the port number and statistical item type.

[0028] In conjunction with the first aspect, in the seventh implementation of the first aspect of the present invention, the shared memory region is mapped to the process address space by calling the shared memory attachment interface based on the system unique key value, and the memory address of the target statistical item in the shared memory region is calculated according to the port number and statistical item type, including:

[0029] The shared memory attachment interface is invoked using the system's unique key value to map the shared memory region to the process address space and obtain the client mapping pointer.

[0030] Based on the offset corresponding to the client mapping pointer, port number, and statistical item type, the memory address of the target statistical item in the shared memory region is calculated.

[0031] In conjunction with the first aspect, in the eighth implementation of the first aspect of the present invention, when the front-end application module receives a statistical data acquisition request, it calls an access interface function to directly read port statistical data from the memory address corresponding to the shared memory region, including:

[0032] When the front-end application module receives a statistical data retrieval request, it parses the port number and statistical item type in the statistical data retrieval request, and calls the access interface function by passing in the port number and the statistical item type;

[0033] The port statistics data of the corresponding data type are read from the memory address returned by the access interface function, and the read port statistics data are returned to the front-end application module.

[0034] Secondly, the present invention provides a real-time data statistics system, the real-time data statistics system comprising:

[0035] The calculation module is used to calculate the total memory space size based on the number of ports on the switch, request a shared memory region through a system unique key value, and map the shared memory region to the process address space;

[0036] A creation module is used to create timer objects. When the timer object's period expires, a callback function is triggered and the read statistical data is written to the shared memory area.

[0037] The reading module is used to directly read port statistics data from the corresponding memory address in the shared memory region by calling the access interface function when the front-end application module receives a statistical data acquisition request.

[0038] The technical solution provided by this invention decouples the collection and access of port statistics data by establishing a shared memory-based data buffering mechanism and combining it with a timer-driven background periodic collection strategy. This transforms the data access target from low-speed hardware registers to high-speed shared memory. When front-end application modules read statistics data, they no longer need to wait for bus communication and chip access; they can directly obtain data from shared memory, reducing access latency from hundreds of milliseconds to microseconds and achieving millisecond-level response speeds. Through the timer-driven background automatic periodic collection mechanism, chip access operations originally triggered by front-end requests are transferred to independent background execution, reducing the frequency of direct access to the exchange chip registers, avoiding multi-process contention for bus resources, lowering CPU utilization, and improving overall performance and stability. All front-end application modules obtain data from the same shared memory region through a unified access interface function, ensuring consistency in data acquisition across different applications and facilitating system expansion. Modules requiring new statistics data only need to connect to the shared memory without modifying the core collection logic. Users experience no latency when performing query operations, resulting in a smooth interactive experience. This solution is suitable for network monitoring scenarios requiring high-frequency refresh of statistical information, effectively improving the user experience.

[0039] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of one embodiment of the real-time data statistics method in this invention;

[0042] Figure 2 This is a schematic diagram of one embodiment of the real-time data statistics system in this invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0045] To facilitate understanding of this embodiment, a real-time data statistics method disclosed in this embodiment of the invention will first be described in detail. For example... Figure 1 As shown, this method includes the following steps:

[0046] 101. Calculate the total memory space size based on the number of ports on the switch, request a shared memory region using the system's unique key value, and map the shared memory region to the process address space;

[0047] 102. Create a timer object. When the timer object's period expires, trigger the callback function and write the read statistics data into the shared memory area.

[0048] 103. When the front-end application module receives a request to obtain statistical data, it calls the access interface function to directly read the port statistical data from the corresponding memory address in the shared memory area.

[0049] In one specific embodiment, the process of performing step 101 may specifically include the following steps:

[0050] Read the physical port configuration information of the switch to obtain the number of ports. Determine the number of statistical items for each port, the data type and storage bytes of each statistical item according to the preset statistical item type list, and calculate the total memory space required to store all statistical items for all ports.

[0051] Generate a system-unique key value, allocate a shared memory region based on the total memory space size and the system-unique key value, and generate a shared memory identifier;

[0052] Based on shared memory identifiers, shared memory regions are mapped to the process address space, port data blocks are divided in order of port number, and offset positions are set according to statistical item type.

[0053] Specifically, the data statistics management module reads the current physical port configuration information of the switch. This configuration information, provided by the system configuration file or driver initialization process, includes the number, enabling status, and function description of each physical port. After parsing the configuration information, the data statistics management module extracts the total number N of active physical ports in the switch. Based on a pre-defined list of statistical item types, it determines the number M of statistical data items that each port needs to maintain. This list includes information across multiple dimensions, such as the number of bytes sent, bytes received, number of packets sent, number of packets received, number of unicast packets, number of multicast packets, number of broadcast packets, number of dropped packets, number of erroneous packets, and number of CRC errors. For each statistical item, its data type and the required number of storage bytes are predefined. For example, byte count items are represented using a 64-bit unsigned integer, occupying 8 bytes, while packet count items use a 32-bit unsigned integer, occupying 4 bytes. Based on the number of ports N and the type and size of the statistical items required for each port, the memory requirement S1 for each port is calculated by accumulating the number of bytes required for each statistical item, and then multiplying this by the total number of ports N to obtain the total memory space S2 used to store all statistical items for all ports. A unique integer key K1 is generated system-wide as the unique identifier of the shared memory region by calling the shared memory interface provided by the operating system. Using this key and S2 as parameters, a shared memory request is initiated to the operating system, creating a logically contiguous shared memory region that is physically managed and scheduled by the operating system, and returning the shared memory identifier K2 of the region. The data statistics management module maps the shared memory region corresponding to K2 to the virtual address space of the current process through a shared memory attach operation and obtains the starting address pointer P1 of the region. The entire shared memory region is linearly divided into N equal port data blocks according to the port number order, with each port data block being S1 bytes in size. The starting address of each port data block is calculated by adding the port number to P1 and multiplying by S1. In each port data block, according to the preset list of statistical items and the corresponding offset strategy, a fixed offset is set for the storage location of each statistical item in the data block. For example, offset 0 to 7 bytes is used to store the number of bytes sent, offset 8 to 15 bytes is used to store the number of bytes received, offset 16 to 19 bytes is used to store the number of messages sent, and so on. Structured access to data is achieved through structure description and offset definition.

[0054] When the number of switch ports or the type of statistics changes, the process also includes dynamic expansion and data migration of shared memory: A configuration information area is reserved at the beginning of the shared memory region to store the current number of ports, a list of statistics types, an offset mapping table for each statistics item, and a memory layout version number. Changes in switch port configuration and statistics types are periodically detected, and the expansion process is triggered when a configuration change is detected. The periodic triggering of timer objects is paused, waiting for the currently executing callback function to complete. The shared memory state is set to maintenance mode to prevent new front-end application module access requests. The total required memory space is recalculated based on the new number of ports and the list of statistics types. A new system-unique key is generated, based on the recalculated... Allocate a new shared memory region based on the total memory space size and obtain a new shared memory identifier. Map the new shared memory region to the process address space. Establish port data block and offset mapping relationships in the new shared memory region according to the new number of ports and statistical item types. Traverse the statistical data of each port in the original shared memory region. Find the corresponding storage location in the new shared memory region according to the port number and statistical item type. Copy the valid statistical data to the corresponding location in the new shared memory region. Initialize the newly added ports or statistical items to zero values. Update the port number, statistical item type list and memory layout version number in the configuration information region. Release the original shared memory region. Restart the timer object to continue collecting data using the new shared memory region.

[0055] In one specific embodiment, the process of mapping a shared memory region to the process address space based on a shared memory identifier, dividing port data blocks according to port number order, and setting offset positions according to statistical item type can specifically include the following steps:

[0056] Based on the shared memory identifier, the shared memory attach interface function is called to map the shared memory region to the process address space, obtain the starting address pointer, and divide the shared memory region into port data blocks according to the port number order;

[0057] Within each port data block, the offset positions of each statistical item are set in a fixed order according to the statistical item type, and a zeroing initialization operation is performed on the shared memory region.

[0058] Specifically, based on the shared memory identifier K1, a shared memory attach operation is invoked to map the corresponding shared memory region to the virtual address space of the process where the current data statistics management module resides, and the starting address pointer P1 of the shared memory region is obtained. According to the total number of physical ports N, the entire shared memory region is divided into N consecutive port data blocks in order of port numbers. The size of each port data block is S1 bytes, and its starting address is calculated by adding the port number to P1 multiplied by S1. This ensures that the logical location of each port data block in the shared memory is clear and non-overlapping. Within each port data block, the order of statistical items is determined according to a predefined list of statistical item types, and the storage offset of each statistical item is sequentially set to a fixed relative position. For example, the number of bytes sent is set in the address range of offset 0 to 7, the number of bytes received is set in the address range of offset 8 to 15, the number of packets sent is set in the address range of offset 16 to 19, and so on, until the offset configuration of all M statistical items is completed. Offset settings follow a unified data structure definition, ensuring that the data blocks of all ports are logically consistent, thereby supporting efficient data access through fixed offset mapping to structures. After all offsets are configured, the shared memory region is zeroed out byte by byte starting from position P1, ensuring that the entire memory region does not contain any historical residual data or undefined values ​​during system initialization.

[0059] After obtaining the starting address pointer, the process also includes steps to avoid data read / write conflicts based on a double-buffer mechanism: Two equal-sized buffers are divided within the shared memory region as a front buffer and a back buffer. Each buffer is divided into port data blocks according to port numbering order, and the offset position of the statistical item is set. The front buffer is initialized to a readable state, and the back buffer to a writable state. A buffer status flag field is set at the beginning of the shared memory region, and the identifier value and version number of the currently readable buffer are stored through atomic operations. The version number is incremented each time the buffer switches. When the timer callback function completes writing statistical data to the back buffer, the buffer status flag field is updated through an atomic comparison and swap operation, setting the back buffer identifier value to a readable state and the front buffer identifier value to a writable state, while simultaneously incrementing the version number. When the front-end application module calls the access interface function, it first reads the buffer status flag field to obtain the current readable buffer identifier value and version number. Based on the readable buffer identifier value, it calculates the memory address of the target statistical item in the readable buffer. After reading, it reads the version number again and compares it with the previous version number. If the version number has not changed, the read data is returned; if the version number has changed, the read operation is re-executed.

[0060] In one specific embodiment, the process of setting the offset position of each statistical item in a fixed order according to the statistical item type within each port data block and performing a zeroing initialization operation on the shared memory region can specifically include the following steps:

[0061] Within each port data block, offset positions for the statistical items of number of bytes sent, number of bytes received, number of packets sent, and number of packets received are assigned in the order of statistical item type.

[0062] Set all bytes in the shared memory region to zero, so that the initial data of each statistical item is zero.

[0063] Specifically, according to the preset order of statistical item types, specific storage offset positions are assigned to each statistical indicator sequentially within each port data block. The number of bytes sent is the first statistical item, with a data type of 64-bit unsigned integer, occupying 8 bytes of storage space, and is allocated at an offset range of 0 to 7 bytes at the beginning of the data block. The number of bytes received is also a 64-bit unsigned integer, stored in the address range of offset 8 to 15 bytes. The number of messages sent is the third statistical item, with a data type of 32-bit unsigned integer, occupying 4 bytes of space, and is therefore allocated at offsets of 16 to 19 bytes. The number of messages received is a 32-bit unsigned integer, occupying 4 bytes, with a corresponding offset range of 20 to 23 bytes. The offset positions are set for each statistical item in the above order, and the offset values ​​of each statistical item are fixed as constants in the memory structure definition. This ensures that any front-end module can calculate a unique corresponding physical address by jointly using the port number and the statistical item type when reading statistical data, achieving deterministic and high-speed data access. After allocating offsets for all statistical items, a zeroing operation is performed on the entire shared memory region, byte by byte. The zeroing process starts from the shared memory starting address P1 and continuously overwrites the entire memory region of size S2 bytes, ensuring that the memory space corresponding to all statistical items within each port data block is zero in the initial state.

[0064] In one specific embodiment, the process of performing step 102 may specifically include the following steps:

[0065] Create a timer object and configure the periodic trigger time, and define a callback function containing the data collection and update logic;

[0066] Register the callback function as the expiration handling function of the timer object, configure the timer object to periodic trigger mode, and call the start interface to activate the timer object;

[0067] When the timer object expires, a callback function is triggered to read the statistical data of all ports in the switching chip in batches through the bus interface, and write the read statistical data into the shared memory area according to the port number and offset position.

[0068] Specifically, during the initialization phase, the data statistics management module creates a periodic data acquisition timer object T1 and configures a fixed periodic trigger time Δt for the timer object. For example, Δt is set to 10 milliseconds to ensure that the statistical data refresh frequency reaches 100 times per second, meeting the high real-time requirements of scenarios such as network monitoring and anomaly detection. While configuring the timer parameters, a callback function F1 is defined, containing complete data acquisition and shared memory update logic. The callback function encapsulates the reading operation of the underlying chip's statistical data, the structured organization of the read results, and the batch update process of writing to shared memory. Each trigger completes a full-port data refresh task. After definition, function F1 is registered as the expiration handling function of timer object T1, establishing a binding relationship between periodic triggering and logical execution. This allows the operating system to automatically schedule and execute F1 after each T1 time period. Timer object T1 is configured for periodic triggering mode, meaning that after executing F1 each time it expires, it automatically enters the next cycle countdown, continuously triggering without manual intervention. After configuration, the timer start interface is called to officially activate T1, causing it to start running at intervals of Δt. From the start time of T1, the timer continuously triggers function F1 to enter the execution state with a period of 10 milliseconds. Each time F1 is called, the system interacts with the switching chip driver module, calling the underlying bus interface function B1 to initiate a data acquisition command using a batch read method. This retrieves the current values ​​of M predefined statistical indicators from all N physical ports of the switching chip at once. The batch read operation is implemented based on bus protocols such as register management bus or serial control bus, using a continuous address segment read mechanism to reduce the number of communication transactions and lower the CPU's interaction load with the bus. After the read is complete, the statistical data returned by the chip is organized into a temporary buffer D1, and each group of port data is processed sequentially according to port number. For port number i, the starting address of its corresponding shared memory data block can be calculated by combining the shared memory start pointer P1 with the number of bytes stored in a single port S1: P1 plus i multiplied by S1. Based on this address, according to the offset constant corresponding to each statistical indicator, the statistical values ​​of the current port in D1 are written to the specified locations in shared memory item by item. For example, the number of bytes sent is written to the offset area from byte 0 to byte 7, the number of bytes received is written to the offset area from byte 8 to byte 15, and so on, until all statistical items are updated. After all port data has been written, the execution of this round of callback function ends, the system returns to the main process, and waits for the trigger signal of the next cycle. Timer T1 continuously drives function F1 to execute with a period of 10 milliseconds, thereby ensuring that the port statistical data in the shared memory area is always up-to-date.

[0069] Before batch reading all port statistics data of the switching chip, a step of hierarchical collection based on port activity is included: maintaining activity evaluation parameters for each port, including data change rate and number of consecutive unchanged times, and initializing the activity evaluation parameters of all ports to default values; when each timer callback function is executed, the current statistics data of each port are first read from the temporary data buffer, and the current statistics data of each port are compared with the historical statistics data of the port in the shared memory area to calculate the data change amount of each port, and update the data change rate and number of consecutive unchanged times of each port based on the data change amount; according to the data change rate and number of consecutive unchanged times of each port, the ports are divided into three levels: high-active ports, medium-active ports, and low-active ports, and a first collection frequency is set for high-active ports, a second collection frequency is set for medium-active ports, and a third collection frequency is set for low-active ports, where the first collection frequency is higher than the second collection frequency, and the second collection frequency is higher than the third collection frequency; when the subsequent timer callback function is executed, according to the activity level and corresponding collection frequency of each port, the statistics data of the corresponding port are selectively read from the switching chip and updated to the shared memory area, and the data of ports that have not reached the collection time are kept unchanged in the shared memory.

[0070] In one specific embodiment, the process of triggering a callback function when the timer object expires, and reading statistical data from all ports in the switching chip in batches through the bus interface, and writing the read statistical data into the shared memory area according to the port number and offset position, can specifically include the following steps:

[0071] When the timer object expires, a callback function is triggered, which calls the underlying driver interface to read all port statistics of the switching chip in batches through the bus, and organizes the read statistics into a temporary data buffer.

[0072] Iterate through the statistical data read from the temporary data buffer, calculate the target address of the shared memory region based on the port number and offset position, and write the read statistical data into the shared memory region based on the target address.

[0073] Specifically, during system operation, when the period Δt of timer object T1 expires, the operating system automatically schedules and triggers the pre-registered callback function F1 to enter the execution state. The main task of the callback function is to complete the collection of statistical data and the update of shared memory data at the current moment. Therefore, inside function F1, the system calls the underlying driver interface D1 that interfaces with the switching chip, issuing a batch read instruction to obtain the current statistical data on all N physical ports from the chip through the underlying control bus protocol. To improve the collection efficiency, the read operation adopts a continuous address segment access method, thereby completing the centralized reading of multiple ports and multiple statistical items in one bus transaction, avoiding performance bottlenecks caused by repeated communication. After responding to the read instruction, the chip driver module returns the statistical data of all current ports to the upper layer module, and the system organizes it into a structured temporary data buffer B1. In the temporary data buffer, each data record consists of a port number, a statistical item identifier, and a corresponding value, arranged in ascending order of port number. The temporary buffer B1 is traversed, reading each data entry one by one. Based on the port number *i* and statistical item type *t* associated with each data entry, the corresponding target write address in shared memory is located. The target address is calculated as follows: using the shared memory start pointer P1 as a base, the product of the port number *i* and the number of bytes occupied by each port *S1* is added to obtain the starting address of the data block for that port in shared memory: A1 = P1 + i × S1. Then, based on the offset constant O1 corresponding to the statistical item type *t*, the relative position of the data entry within the port data block is calculated to obtain the precise data write address within the shared memory region: A2 = A1 + O1. Based on address A2, the statistical data value of the current entry is written to the shared memory location corresponding to that address, and this process is repeated until all entries in buffer B1 have been traversed.

[0074] After writing the read statistical data into the shared memory area, the process includes steps for anomaly detection and early warning based on a threshold model: A dynamic threshold model is established for each statistical item of each port. This model includes the mean, standard deviation, maximum growth rate, and continuous anomaly counter for the statistical item within a sliding time window. The length parameter of the sliding time window and the anomaly judgment coefficient are initialized. After each statistical data is written into the shared memory area, the change and rate of change of the current statistical data relative to the previous statistical data are calculated. The change and rate of change are compared with the mean and standard deviation in the dynamic threshold model. When the rate of change exceeds the mean plus the anomaly judgment coefficient times the standard deviation, it is judged as a suspected anomaly. For statistical items judged as suspected anomalies, their corresponding continuous anomaly counter is incremented. When the continuous anomaly counter reaches a preset confirmation threshold, it is confirmed as a real anomaly, and an anomaly event record is generated. The anomaly event record includes the port number, statistical item type, anomaly value, anomaly timestamp, and anomaly type identifier. The anomaly event record is written into a separate anomaly log area in the shared memory area, and an anomaly flag is set to notify the front-end application module. The front-end application module reads the anomaly log area through an access interface function to obtain the anomaly event record and performs alarm processing. Simultaneously, the mean and standard deviation of the dynamic threshold model are updated based on the latest statistical data.

[0075] After configuring the periodic trigger time, the process also includes dynamically adjusting the timer period based on port statistics change characteristics: setting a base period parameter and an adjustable period range for the timer object. The base period parameter is the initially configured fixed period time, and the adjustable period range limits the minimum and maximum values ​​of the timer period. A global data change rate index is maintained in the shared memory area, which records the average change amplitude of all port statistics within a unit of time. Each time the timer callback function is executed, the current port statistics read from the switching chip are compared item by item with the historical statistics in the shared memory area. The number of ports and statistical items that have changed values ​​are counted, and the total change of all changed statistical items is calculated. Based on the proportion of the number of changed ports to the total number of ports and the change in value, the total change in value is calculated. The system calculates the global data change rate for the current period by summing the total data. It then averages the global data change rates over multiple consecutive collection periods using a sliding window. When the average change rate exceeds a preset high-flow threshold, it indicates the switch is under high load. In this case, the timer period is adjusted towards the minimum of the adjustable period range to increase the collection frequency. Conversely, when the average change rate falls below a preset low-flow threshold, it indicates the switch is under low load. In this case, the timer period is adjusted towards the maximum of the adjustable period range to decrease the collection frequency. The system updates the timer object's period trigger time to the adjusted period value via the timer configuration interface, ensuring subsequent callback functions execute according to the new period. The system records the current timer period value in the configuration information area of ​​the shared memory region for query by the front-end application module to assess the real-time performance of the data.

[0076] In one specific embodiment, after writing the read statistical data to the shared memory region, the method further includes:

[0077] Define the access interface function and set the port number and statistical item type as input parameters, and validate the input parameters.

[0078] The shared memory attachment interface is called based on the system's unique key value to map the shared memory region to the process address space, and the memory address of the target statistical item in the shared memory region is calculated based on the port number and statistical item type.

[0079] Specifically, the data statistics management module defines a unified data access interface function F2, and passes the port number parameter P1 and the statistical item type parameter T1 as input variables to the function execution logic. Inside function F2, to ensure that the requested statistical data has a clear logical direction and does not access illegal memory regions out of bounds, the input parameters undergo strict validity verification. Specifically, for the port number P1, it is determined whether it falls within the valid port number range [0, N]. Within [1], the value must be no less than 0 and no greater than or equal to the total number of physical ports N. For statistical item type T1, it is checked whether it is a valid element in the predefined statistical item enumeration set, including common statistical item names such as the number of bytes sent, the number of bytes received, the number of messages sent, and the number of messages received. Any type not in the valid enumeration set is rejected, thus avoiding access to uninitialized or undefined memory offset regions. After the input parameters are verified, the globally unique integer key value K1 is used as the positioning basis. The shared memory attachment operation is called to map the corresponding shared memory region to the user space of the process where the current function is executing, and the starting address pointer P2 of the shared memory region is obtained. Based on the parameter P1 and the fixed number of bytes S1 occupied by each port data block, the starting address A1 of the target port data block is calculated. The calculation formula is A1 = P2 + P1 × S1, which can accurately locate the data region corresponding to port P1. According to the parameter T1, the offset constant O1 corresponding to this type is looked up in the internal statistical item offset mapping table. This offset represents the storage location of the current statistical item in a single port data block. By adding A1 and O1, i.e., A2 = A1 + O1, we obtain the unique valid address A2 of the currently requested statistics item in the shared memory region. Based on the data type corresponding to T1 (such as a 64-bit unsigned integer or a 32-bit unsigned integer), we read the actual stored statistical data value from address A2 and pass this value as the return result of function F2 to the caller.

[0080] In one specific embodiment, the process of executing the shared memory attachment interface based on the system unique key value to map the shared memory region to the process address space, and calculating the memory address of the target statistical item in the shared memory region according to the port number and statistical item type, can specifically include the following steps:

[0081] The shared memory attach interface is called using the system's unique key-value pair to map the shared memory region to the process address space and obtain the client's mapping pointer.

[0082] Based on the client-mapped pointer, port number, and offset corresponding to the statistical item type, calculate the memory address of the target statistical item in the shared memory region.

[0083] Specifically, using the globally unique integer key K1 generated by the data statistics management module during the initialization phase, the shared memory attachment interface provided by the operating system is called to map the shared memory region corresponding to this key value to the virtual address space of the current client process. After attachment, the starting mapping pointer P3 of the shared memory region on the client side is obtained. The completion of the mapping operation indicates that the client process has read-only access to the shared memory region and can directly obtain the latest value of the data statistics item by reading from local memory, without having to access the underlying hardware through kernel-mode drivers or bus interfaces, thus significantly reducing read latency. Based on the client mapping pointer P3, according to the port number parameter P1 provided by the user input or module call, combined with the single-port data block size S1 that has been uniformly defined during the system initialization phase, the starting address A1 of the data block corresponding to the target port in the shared memory region is obtained by calculating the expression P3 plus P1 multiplied by S1. The starting address of the data block is the unified storage starting point for all statistics items of the current port. According to the requested statistics item type parameter T1, the corresponding offset constant O1 is retrieved from the predefined statistics item offset mapping table. This offset value represents the storage location of the statistics item relative to the port data block starting address A1. Add A1 and O1 together, i.e., A2 = A1 + O1, to obtain the final absolute address A2 of the target statistic in the shared memory region.

[0084] In one specific embodiment, the process of performing step 103 may specifically include the following steps:

[0085] When the front-end application module receives a statistical data retrieval request, it parses the port number and statistical item type in the statistical data retrieval request, and calls the access interface function by passing in the port number and statistical item type.

[0086] The port statistics data of the corresponding data type are read from the memory address returned by the access interface function, and the read port statistics data is returned to the front-end application module.

[0087] Specifically, the request for statistical data is parsed to extract the requested port number parameter P1 and the corresponding statistical item type parameter T1. These parameters undergo a preliminary format validity check to ensure they meet the requirements of the valid range and data structure. After parsing the request parameters, the front-end application module calls the data access interface function F2, predefined by the data statistics management module, passing the extracted port number P1 and statistical item type T1 as input parameters to trigger the background shared memory data access process. Inside the access interface function F2, a shared memory attach operation is called based on the globally unique key K1 to map the shared memory region to the virtual address space of the process to which the current front-end module belongs. The mapped starting pointer P3 is obtained, ensuring that all subsequent access operations are calculated based on local addresses, without relying on any chip access instructions or kernel-mode resource allocation. Based on the input parameter P1 and the known fixed number of bytes S1 occupied by each port's data block, the starting address A1 of the data block for that port in shared memory is calculated using the formula A1 = P3 + P1 × S1, ensuring that the entire data region of the target port can be located. Based on this, the system looks up the fixed offset O1 corresponding to the statistical item type T1 in a table, and calculates the final absolute address A2 of the statistical item in shared memory using the formula A2 = A1 + O1. After address location, the system reads data content of the same data type as T1 from address A2. For example, when T1 represents the number of bytes sent or received, the system reads a 64-bit unsigned integer from address A2; when T1 represents the number of packets sent or received, it reads a 32-bit unsigned integer. The system then performs format encapsulation and boundary checks on the read data to ensure that the output data conforms to the type specification, is numerically valid, and is complete. After the data is read, the access interface function F2 returns the statistical data as a return value to the front-end calling module. Upon receiving the return value, the front-end module sends it back to the user interface or upper-layer calling flow as a response, thus realizing the complete query process of port statistics information.

[0088] When the front-end application module needs to retrieve multiple ports or multiple statistical items, the process also includes batch reading and local caching optimization steps: The extended access interface function supports batch reading mode, adding batch request parameters. These parameters include multiple port numbers and a list of corresponding statistical item types. A local cache data structure is maintained within the front-end application module, storing recently accessed port statistics and their timestamps. When the front-end application module receives a batch statistics retrieval request, it first checks the local cache data structure. For statistical items that exist in the cache and whose timestamps are valid, the data is directly returned from the local cache. Statistical items that do not exist in the cache or have expired are compiled into a retrieval list. For the statistical items in the retrieval list, the extended access interface function is called, passing in the batch request parameters. Inside the access interface function, a shared memory attachment operation is completed in one go. The port numbers and statistical item types in the batch request parameters are traversed, and the memory addresses of each statistical item in the shared memory area are calculated in batches and data is read continuously. After completion, a shared memory separation operation is performed. The statistical data obtained from the batch reading is updated in the local cache data structure, and the timestamp is updated. The local cache data and the batch read data are merged, and the complete statistical data result is returned to the front-end application module.

[0089] When multiple front-end application modules access the system concurrently, a priority-based access scheduling and rate limiting mechanism is also included: Access priorities are assigned to different types of front-end application modules, including command-line interface modules, network management interface modules, monitoring agent modules, and internal diagnostic modules. Internal diagnostic modules have the highest priority, followed by command-line interface modules, and then network management interface modules and monitoring agent modules have the lowest priority. An access control area is set up in the shared memory region to store the access priorities and rate limiting parameters of each module. A caller identification parameter is added to the access interface function. When a front-end application module calls the access interface function, it passes its own module type identifier. The access interface function reads the corresponding access priority and rate limiting parameters from the access control area based on the module type identifier. The rate limiting parameters include a time window. The system sets the maximum number of accesses and the minimum access interval within the shared memory area. For each front-end application module, an access counter and a last access timestamp are maintained. When an access interface function is called, the system checks whether the caller's access count within the current time window exceeds the maximum number of accesses and whether the time interval since the last access is less than the minimum access interval. If the limit is exceeded, the system processes requests based on priority. For low-priority modules exceeding the rate limit, a rate-limiting error code is returned, and the user is asked to wait. For high-priority modules exceeding the rate limit, access is allowed to continue beyond the limit, but an alarm log is recorded. For access requests within the normal range, the access counter and last access timestamp are updated, and the normal data reading process is executed. The access counters for each module are periodically reset to open new time windows.

[0090] The above describes the real-time data statistics method in the embodiments of the present invention. The following describes the real-time data statistics system in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the real-time data statistics system in this invention includes:

[0091] The calculation module 201 is used to calculate the total memory space size based on the number of ports of the switch, request a shared memory region through the system unique key value, and map the shared memory region to the process address space;

[0092] Create module 202 to create timer objects. When the timer object's period expires, a callback function is triggered and the read statistics are written to the shared memory area.

[0093] The reading module 203 is used to directly read port statistics data from the corresponding memory address in the shared memory region by calling the access interface function when the front-end application module receives a statistical data acquisition request.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time data statistics method, characterized in that, include: The total memory space size is calculated based on the number of ports on the switch, a shared memory region is requested using a system-unique key value, and the shared memory region is mapped to the process address space. A timer object is created. When the timer object's period expires, a callback function is triggered, and the read statistical data is written to the shared memory area. Specifically, this includes: creating a timer object and configuring the periodic trigger time; defining a callback function containing data acquisition and update logic; registering the callback function as the timer object's expiration handling function; configuring the timer object in periodic trigger mode and calling the startup interface to activate the timer object; when the timer object's period expires, triggering the callback function; calling the underlying driver interface to read all port statistical data of the switching chip in batches via the bus; organizing the read statistical data into a temporary data buffer; and iterating through the temporary data buffer to read... The statistical data is processed by calculating the target address of the shared memory region based on the port number and offset position, and then writing the read statistical data into the shared memory region based on the target address. After writing the read statistical data into the shared memory region, the process further includes: defining an access interface function and setting the port number and statistical item type as input parameters, and validating the input parameters; using the system unique key value to call the shared memory attachment interface to map the shared memory region to the process address space and obtain a client mapping pointer; and calculating the memory address of the target statistical item in the shared memory region based on the client mapping pointer, the port number, and the offset corresponding to the statistical item type. When the front-end application module receives a request to obtain statistical data, it calls the access interface function to directly read the port statistical data from the memory address corresponding to the shared memory region.

2. The real-time data statistics method according to claim 1, characterized in that, The total memory space size is calculated based on the number of ports on the switch. A shared memory region is requested using a system-unique key value, and the shared memory region is mapped to the process address space, including: Read the physical port configuration information of the switch, obtain the number of ports, determine the number of statistical items for each port and the data type and storage bytes of each statistical item according to the preset statistical item type list, and calculate the total memory space required to store all statistical items for all ports; Generate a system-unique key value, and based on the total memory space size and the system-unique key value, apply for a shared memory region and generate a shared memory identifier; Based on the shared memory identifier, the shared memory region is mapped to the process address space, the port data blocks are divided according to the port number order, and the offset position is set according to the statistical item type.

3. The real-time data statistics method according to claim 2, characterized in that, Based on the shared memory identifier, the shared memory region is mapped to the process address space, and port data blocks are divided according to port number order and offset positions are set according to statistical item type, including: Based on the shared memory identifier, the shared memory attach interface function is called to map the shared memory region to the process address space, obtain the starting address pointer, and divide the shared memory region into port data blocks according to the port number order; Within each port data block, the offset positions of each statistical item are set in a fixed order according to the statistical item type, and a zeroing initialization operation is performed on the shared memory region.

4. The real-time data statistics method according to claim 3, characterized in that, Within each port data block, the offset positions of each statistical item are set in a fixed order according to the statistical item type, and a zeroing initialization operation is performed on the shared memory region, including: Within each port data block, offset positions for the statistical items of number of bytes sent, number of bytes received, number of packets sent, and number of packets received are assigned in the order of statistical item type. Set all bytes in the shared memory region to zero, so that the initial data of each statistical item is zero.

5. The real-time data statistics method according to claim 1, characterized in that, When the front-end application module receives a request to retrieve statistical data, it calls the access interface function to directly read the port statistical data from the corresponding memory address in the shared memory region, including: When the front-end application module receives a statistical data acquisition request, it parses the port number and statistical item type in the statistical data acquisition request, and calls the access interface function by passing in the port number and the statistical item type. The port statistics data of the corresponding data type are read from the memory address returned by the access interface function, and the read port statistics data are returned to the front-end application module.

6. A real-time data statistics system, characterized in that, A method for performing real-time data statistics as described in any one of claims 1-5, comprising: The calculation module is used to calculate the total memory space size based on the number of ports on the switch, request a shared memory region through a system unique key value, and map the shared memory region to the process address space; A creation module is used to create timer objects. When the timer object's period expires, a callback function is triggered and the read statistical data is written to the shared memory area. The reading module is used to directly read port statistics data from the corresponding memory address in the shared memory region by calling the access interface function when the front-end application module receives a statistical data acquisition request.

Images