Traffic statistics method and device, electronic equipment and storage medium

Abstract

The application provides a traffic statistics method, which comprises the following steps: initializing a traffic statistics process, at least comprising memory pool initialization based on first data created in shared memory, message transmission initialization; establishing a service application connection; and performing traffic statistics on the service application in a data transmission process by the traffic statistics process. The application also provides a traffic statistics device, system, electronic equipment and readable storage medium.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a flow statistics method, apparatus, system, electronic device and storage medium. Background Technology

[0002] Network security equipment provides secure transmission and inspection functions for various business data, as well as query functions for various business traffic information. The equipment's web management system allows for real-time querying and display of business traffic, enabling operators to understand the operational status of each business within the system. Traffic data for each business is presented in the form of line graphs, pie charts, bar charts, etc., making the data more intuitive for operators. Therefore, statistical analysis of business traffic is essential.

[0003] Existing technologies can be implemented as follows: The business application process performs traffic statistics and records them within the business connection while processing business messages. The traffic statistics data is then periodically sent to the traffic statistics process via network communication. The traffic statistics process then starts a listening service to read the received business traffic statistics data and write it to a database. Alternatively, the existing technologies can be implemented as follows: The business application process performs traffic statistics and records them within the business connection while processing business messages. The traffic statistics data is then periodically added to a cache queue in shared memory as memory blocks. The traffic statistics process periodically reads the memory blocks in the cache queue and writes the data to a database.

[0004] The problems with the existing technology are as follows: first, it cannot count traffic in real time; second, the increase in business data volume and business connection concurrency will lead to increased network connection overhead, affecting the overall system performance. Summary of the Invention

[0005] In order to solve at least one of the above-mentioned technical problems in the prior art, the purpose of this application is to provide a traffic statistics method, apparatus, electronic device and storage medium.

[0006] According to the traffic statistics method provided in the first aspect of this application, the method includes:

[0007] Initialize the traffic statistics process, which includes at least initializing the memory pool and message transmission based on the first data created in shared memory.

[0008] Establish business application connections based on business application processes to generate business application data traffic; and

[0009] The traffic statistics process is used to collect statistics on the traffic of the business application during data transmission.

[0010] According to one embodiment of this application, the first data includes a lock-free circular queue array;

[0011] The memory pool initialization includes:

[0012] A first lock-free circular queue array is created in shared memory to store the memory block management structure of the memory blocks required by the business application. The index of the first lock-free circular queue array corresponds one-to-one with the CPU core number.

[0013] The member data of the memory block management structure includes at least the business application address, which is used to identify the address pointer of the memory block required by the business application.

[0014] The message transmission initialization according to one embodiment of this application includes:

[0015] A second lock-free circular queue array is created in shared memory, which is used to dynamically write the memory block addresses stored in the first lock-free circular queue array. The index of the second lock-free circular queue array corresponds one-to-one with the CPU core number.

[0016] According to one embodiment of this application, the initialization of the traffic statistics process further includes data cache initialization, which includes:

[0017] Create a singly linked list, which is used to cache pointer addresses of memory blocks required by the business application, obtained from the second lock-free circular queue array; and

[0018] Start a timer to periodically traverse the singly linked list and periodically obtain the pointer addresses of the memory blocks required by the business application from the second lock-free circular queue array.

[0019] According to one embodiment of this application, the initialization of the traffic statistics process further includes category summary initialization, which includes: creating a hash table, the hash table being used to record traffic information generated by multiple different business applications during traffic statistics.

[0020] According to one embodiment of this application, establishing a business application connection based on a business application process includes:

[0021] Allocate the memory blocks required by the business application;

[0022] Save the pointer addresses of the memory blocks required by the business application to the business connection;

[0023] The memory block required by the business application is initialized based on the first business message data of the business application;

[0024] Write the pointer address of the memory block required by the business application into the first lock-free circular queue array;

[0025] The data caching module traverses the second lock-free circular queue array of the message transmission module, reads the pointer address of the memory block required by the business application, and saves it to the singly linked list.

[0026] According to one embodiment of this application, the initialization of the memory block required by the service application based on the first service message data of the service application includes:

[0027] Based on the first business message data of the business application, the memory blocks required by the business application are initialized through the memory block management structure;

[0028] The member data of the memory block management structure also includes the IP address, port, application protocol, traffic statistics value and Flag of the business application. The Flag is used to indicate whether the memory block required by the business application is occupied.

[0029] The initialization of the memory block required by the business application based on the first business message data of the business application through the memory block management structure includes: obtaining the IP address, port, and application protocol of the business application based on the first business message data of the business application, assigning the IP address, port, and application protocol of the business application to the corresponding member data of the memory block management structure, initializing the traffic statistics value to zero, and initializing the Flag to indicate that the memory block required by the business application is occupied.

[0030] According to one embodiment of this application, the traffic statistics process is used to count the traffic of the service application during data transmission, including:

[0031] When processing data packets after the first business message, the business application process accesses the memory block management structure through the pointer address of the memory block required by the business application stored in the business connection, and updates the traffic statistics value of the memory block management structure with the business traffic data;

[0032] The system periodically accesses the singly linked list and reads the data of the business application through the pointer addresses of the memory blocks required by the business application in the singly linked list; and

[0033] The data from the aforementioned business applications is categorized, recorded using a hash table, and traffic is aggregated and statistically analyzed according to different categories.

[0034] According to one embodiment of this application, the method further includes the business application closing the connection, wherein the business application closing the connection includes:

[0035] The member data of the memory block management structure also includes a timestamp marker, which is used to identify the start and end times of the business application traffic statistics;

[0036] When the business application process receives a data packet to close the connection, it accesses the memory block management structure through the pointer address of the memory block required by the business application in the business connection, updates the Flag in the memory block management structure to the end flag, and updates the timestamp flag.

[0037] According to one embodiment of this application, the method further includes releasing resources, the resource release comprising:

[0038] Query the second lock-free circular queue, read the pointer address of the memory block required by the business application, and write it into the singly linked list;

[0039] Traverse the singly linked list and read the data of the memory block required by the business application, send the data of the memory block required by the business application to the classification and summarization module, and remove the pointer address of the memory block required by the business application from the singly linked list;

[0040] Based on the data in the memory blocks required by the business application, the statistical data in the hash table is categorized and statistically analyzed, written to the database, and then the hash table is cleared; and

[0041] Release the memory blocks required by the business applications in the memory pool.

[0042] According to the flow statistics device provided in the second aspect of this application, the device includes:

[0043] The initialization module initializes the traffic statistics process, including at least memory pool initialization based on the first data created in shared memory and message transmission initialization.

[0044] The business application process module is used to establish business application connections based on business application processes to generate business application data traffic; and

[0045] The business traffic statistics module is used to perform traffic statistics on the business application through the traffic statistics process.

[0046] According to the traffic statistics system provided in the third aspect of this application, the traffic statistics system includes:

[0047] The business application process module is used to establish business application connections to generate business application data traffic.

[0048] A traffic statistics process module is used to count the data traffic of the business application based on the initialized traffic statistics process to obtain traffic statistics information; the initialized traffic statistics process includes at least initializing the memory pool based on the first data created in shared memory and initializing message transmission; a management system is used to enable users to query the traffic statistics information of the business application through human-computer interaction based on the traffic statistics information; and

[0049] A database is used to store the traffic statistics information.

[0050] An electronic device according to a fourth aspect of this application includes a processor and a memory, wherein the memory stores an executable program, and the memory executes the executable program to perform the traffic statistics method described in any of the preceding claims.

[0051] According to the storage medium provided in the fifth aspect of this application, the storage medium carries one or more computer programs, which, when executed by a processor, implement the traffic statistics method described in any of the above claims.

[0052] The traffic statistics method provided in this application, on the one hand, can perform real-time statistics on business application data traffic by caching data in shared memory, and on the other hand, can meet the needs of complex business statistics by classifying and summarizing business data packets. Attached Figure Description

[0053] Figure 1 This is a schematic flowchart of a traffic statistics method according to an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of a flow statistics device according to an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of the traffic statistics system structure according to an embodiment of this application.

[0056] Figure label:

[0057] 2001. Initialization Module; 2002. Business Application Process Module; 2003. Business Traffic Statistics Module; 1002. Traffic Statistics Process Module; 1003. Memory Pool Module; 1004. Message Transmission Module; 1005. Data Cache Module; 1006. Classification and Summary Module; 1100. Bus; 1200. Processor; 1300. Memory; 1400. Other Circuits. Detailed Implementation

[0058] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0059] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0060] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0061] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0062] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0063] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0064] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0065] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0066] Figure 1 A schematic flowchart of a traffic statistics method according to one embodiment of this application is shown, as follows: Figure 1 As shown, traffic statistics methods may include:

[0067] S101. Initialize the traffic statistics process, which includes at least initializing the memory pool and message transmission based on the first data created in shared memory.

[0068] S102. Establish a business application connection based on the business application process to generate business application data traffic;

[0069] S103. The traffic statistics process is used to collect statistics on the traffic of business applications during data transmission.

[0070] The traffic statistics method provided in this application can perform real-time statistics on business application data traffic by caching data in shared memory. At the same time, by classifying and summarizing the data of business applications, it meets the needs of complex business statistics.

[0071] According to one embodiment of this application, the first data includes a lock-free circular queue array, and the memory pool initialization in step S101 above may include:

[0072] Create a first lock-free circular queue array in shared memory to store the memory block management structure required by the business application. The index of the first lock-free circular queue array corresponds one-to-one with the CPU core number.

[0073] The memory block management structure includes at least the business application address as a member data, which is used to identify the address pointer of the memory block required by the business application.

[0074] According to one embodiment of this application, the message transmission initialization in step S101 above may include:

[0075] A second lock-free circular queue array is created in shared memory. This array is used to dynamically write the memory block addresses stored in the first lock-free circular queue array. The indices of the second lock-free circular queue array correspond one-to-one with the CPU core numbers.

[0076] According to one embodiment of this application, the initialization of the traffic statistics process in step S101 may further include data cache initialization, which may include: creating a singly linked list and starting a timer. The singly linked list is used to cache the pointer addresses of the memory blocks required by the business application obtained from the second lock-free circular queue array, and the timer is used to periodically traverse the singly linked list and periodically obtain the pointer addresses of the memory blocks required by the business application from the second lock-free circular queue array.

[0077] According to one embodiment of this application, the initialization of the traffic statistics process in step S101 may further include classification and summary initialization, which may include: creating a hash table for storing traffic information generated by multiple different business applications during traffic statistics.

[0078] This application manages the memory block information required by business applications through a lock-free circular queue array, including a first lock-free circular queue array and a second lock-free circular queue array, and then performs traffic statistics on business applications based on this information. This application implements the organization and management of memory block information in a lock-free communication manner, which improves the utilization efficiency of memory blocks required for traffic statistics on the one hand, and effectively improves the performance of business traffic statistics on the other hand.

[0079] According to one embodiment of this application, the step S102 above, which establishes a business application connection based on a business application process, may include:

[0080] S1021. Allocate memory blocks required by the business application for the business application;

[0081] S1022. Save the pointer address of the memory block required by the business application to the business connection;

[0082] S1023. Initialize the memory block required by the business application based on the first business message data of the business application;

[0083] S1024. Write the pointer addresses of the memory blocks required by the business application into the first lock-free circular queue array;

[0084] S1025. The data caching module traverses the second lock-free circular queue array of the message transmission module, reads the pointer address of the memory block required by the business application, and saves it to a singly linked list.

[0085] According to one embodiment of this application, step S1023, which initializes the memory block required by the business application based on the first business message data of the business application, includes:

[0086] Based on the first business message data of the business application, the memory blocks required by the business application are initialized through the memory block management structure;

[0087] The member data of the memory block management structure also includes the IP address, port, application protocol, traffic statistics, and Flag flag of the business application. The Flag flag is used to indicate whether the memory block required by the business application is occupied.

[0088] Based on the first business message data of the business application, the memory block required by the business application is initialized through the memory block management structure. This includes: obtaining the IP address, port, and application protocol of the business application based on the first business message data of the business application, assigning the IP address, port, and application protocol of the business application to the corresponding member data of the memory block management structure, initializing the traffic statistics value to zero, and initializing the Flag to indicate that the memory block required by the business application is occupied.

[0089] According to one embodiment of this application, step S103, which involves statistically analyzing the traffic of a service application during data transmission through a traffic statistics process, may include:

[0090] S1031. When the business application process processes the data packets after the first business packet, it accesses the memory block management structure through the pointer address of the memory block required by the business application stored in the business connection, and updates the business traffic data to the traffic statistics value of the memory block management structure.

[0091] S1032. Periodically access the singly linked list and read the data of the business application through the pointer address of the memory block required by the business application in the singly linked list;

[0092] S1033. Classify the data of business applications, record it through a hash table, and summarize and statistically analyze the traffic according to different categories.

[0093] In step S1033 above, each record in the hash table includes a data pair consisting of a key and a value. The business application ID can be used as the key, and the application aggregate traffic can be used as the value for aggregate statistics.

[0094] According to one embodiment of this application, the traffic statistics method may further include: S104, the service application closes the connection:

[0095] The member data of the aforementioned memory block management structure may also include timestamps, which are used to identify the start and end times of traffic statistics for business applications.

[0096] The business application closing the connection includes: when the business application process receives the data packet to close the connection, it accesses the memory block management structure through the pointer address of the memory block required by the business application in the business connection, updates the Flag in the memory block management structure to the end flag, and updates the timestamp flag.

[0097] According to one embodiment of this application, the traffic statistics method further includes: S105, releasing resources, wherein releasing resources may include:

[0098] S1051. Query the second lock-free circular queue, read the pointer address of the memory block required by the business application and write it into the singly linked list;

[0099] S1052. Traverse the singly linked list and read the data of the memory blocks required by the business application, classify and count the data of the memory blocks required by the business application, write the statistical data in the hash table into the database, and clear the hash table.

[0100] S1053. Remove the pointer address of the memory block required by the business application from the singly linked list;

[0101] S1054. Release the memory blocks required by the business applications in the memory pool.

[0102] A flow rate counting device 2000 according to one embodiment of this application includes:

[0103] Initialization module 2001 initializes the traffic statistics process, including at least initializing the memory pool based on the first data created in shared memory and initializing message transmission.

[0104] Business application process module 2002 is used to establish business application connections based on business application processes to generate business application data traffic;

[0105] The Business Traffic Statistics Module 2003 is used to perform traffic statistics on business applications through the traffic statistics process.

[0106] It should be noted that the implementation details of each module of the flow statistics device 2000 provided in this embodiment correspond to this application. Figure 1 The traffic statistics method for the implementation method will not be described in detail here.

[0107] Figure 3 This is a traffic statistics system according to one embodiment of the present application. The traffic statistics system may include a business application process module 1001, a traffic statistics process module 2002, a web management system 1007, and a database 1008. The business application process module 1001 is used to establish business application connections to generate business application data traffic. The traffic statistics process module 2002 is used to count the data traffic of the business application based on the initialization of the traffic statistics process to obtain traffic statistics information. The initialization of the traffic statistics process includes at least initializing a memory pool based on first data created in shared memory and initializing message transmission. The management system 1007 is used to enable users to query the traffic statistics information of the business application through human-computer interaction based on the traffic statistics information. The database 1008 is used to store the traffic statistics information. The traffic statistics process module 2002 includes a memory pool module 1003, a message transmission module 1004, a data caching module 1005, and a classification and summarization module 1006.

[0108] According to one embodiment of this application, the memory pool module 1003 can provide an access interface for allocating and returning memory blocks required by business applications in shared memory, so that other modules or external processes can call the access interface.

[0109] According to one embodiment of this application, a memory pool data structure is used to access memory block information required by business applications. It is initialized when the system starts up and pre-allocates the required memory blocks in shared memory. The memory pool data structure can be, for example, a memory block management structure, which manages the memory blocks required by business applications.

[0110] According to one embodiment of this application, the memory pool module 1003 is configured to create a first lock-free circular queue array in shared memory. This first lock-free circular queue array stores a memory block management structure for memory blocks required by business applications. The indices of the first lock-free circular queue array correspond one-to-one with the CPU core numbers. The memory block management structure manages the multi-core, multi-process memory blocks required by the application business. The member data of the memory block management structure includes at least the business application address, which is used to identify the address pointer of the memory block required by the business application. The memory block management structure may also include member data such as Flag, IP address, port, application protocol, CPU core number, and traffic statistics. The Flag is used to indicate whether the memory block is occupied. A lock-free processing method with multiple producers and multiple consumers is adopted when allocating and reading memory blocks.

[0111] According to one embodiment of this application, the memory pool module 1003 is further configured to: traverse all memory blocks required by business applications, obtain the memory block management structure stored in the memory blocks, determine the status of the memory blocks required by the business applications based on the management structure, and obtain the traffic statistics value of the memory blocks. The aforementioned steps of traversing all memory blocks required by business applications, obtaining the memory block management structure stored in the memory blocks, determining the status of the memory blocks required by the business applications based on the management structure, and obtaining the traffic statistics value of the memory blocks can be provided to non-external processes via an interface. For example, the memory pool module 1003 can provide a query interface supporting the above functions, allowing external processes or other modules to query the status or traffic statistics value of the memory blocks required by the business applications by calling this query interface.

[0112] According to one embodiment of this application, the message transmission module 1004 is used to transmit the memory block pointer address allocated by the business application process module 1001 from the memory pool module 1003 to the data cache module 1005.

[0113] According to one embodiment of this application, the message transmission module 1004 can be configured to create a second lock-free circular queue array, with the array index corresponding one-to-one with the CPU core number. The message transmission module 1004 can be used to transmit memory block pointers of multi-core and multi-process systems.

[0114] According to one embodiment of this application, the message transmission module 1004 is further configured to write the memory block pointer address required by the business application into the second lock-free circular queue array, or further configured to read the memory block pointer address required by the business application from the second lock-free circular queue array. The aforementioned writing or reading of the memory block pointer address required by the business application into the second lock-free circular queue array can also be provided to other modules or external processes via an interface. For example, a write interface can be provided to write the memory block pointer address required by the business application into the second lock-free circular queue array, and a read interface can be provided to read the memory block pointer address required by the business application from the second lock-free circular queue array.

[0115] According to one embodiment of this application, the business application process module 1001, acting as a producer process, writes the memory block pointer address allocated from the memory pool module 1003 to the first lock-free circular queue through the write interface of the message transmission module 1004. When calling the write interface, the CPU core number where the business resides is passed in, and the memory block pointer address is written to the lock-free circular queue corresponding to the CPU core. The message transmission module 1004 of the traffic statistics process transmits the memory block pointer address to the data cache module through the lock-free circular queue. The traffic statistics process, acting as a consumer process, reads the memory block pointer address from the lock-free circular queue through the read interface of the data cache module 1004.

[0116] According to one embodiment of this application, the producer process 2002 of the business traffic statistics process module calls the allocation interface provided by the memory pool module 1003 to allocate a memory block from the memory pool, and stores the pointer address of the memory pool in the business application process module 1001 for storing business traffic statistics. When calling the allocation interface, the CPU core number where the business is located is passed in, and the memory block is obtained from the memory pool corresponding to the CPU core.

[0117] According to one embodiment of this application, the memory pool module 1003 provides a return interface for returning memory blocks required by business applications to the memory pool. When calling the return interface, the CPU core number where the business is located is passed in, and the memory blocks are returned from the memory pool corresponding to the CPU core.

[0118] According to one embodiment of this application, when the business traffic statistics process module 2002 stops service, it can call the release interface provided by the memory pool module 1003 to release the memory blocks required by the business applications in the memory pool.

[0119] According to one embodiment of this application, the data caching module 1005 caches the memory block pointer addresses required by the acquired business application in shared memory, and periodically reads the traffic data in the memory blocks. The data caching module 1005 can use a singly linked list data structure to cache the memory block pointer addresses.

[0120] According to one embodiment of this application, the data caching module 1005 periodically traverses the lock-free circular queue in the message transmission module to obtain the memory block pointer address, and caches the memory block pointer address in a singly linked list.

[0121] According to one embodiment of this application, the data caching module 1005 periodically traverses the singly linked list and reads the memory block data. When the Flag in the memory block management structure is marked as the end flag, the memory block pointer is removed from the singly linked list, and the memory block is returned to the memory pool by calling the return interface provided by the memory pool module 1003.

[0122] According to one embodiment of this application, the data caching module 1005 periodically traverses the singly linked list to read memory block data, and calls the interface provided by the classification and statistics module 1006 to send the memory block data to the classification and statistics module 1006, which then classifies, summarizes, and statistically analyzes it.

[0123] According to one embodiment of this application, the classification and summarization module 1006 is used to obtain data on memory blocks required by business applications and to summarize and statistically analyze the data according to different business requirement categories. Specifically, the classification and summarization module 1006 can obtain the data on memory blocks required by business applications through an interface.

[0124] According to one embodiment of this application, the classification and summarization module 1006 can be configured to perform traffic statistics on the data generated by the business application according to the attribute type of the business application. The attribute type may be, for example, the business application source IP, the business application destination IP, the business application source port, the business application destination port, the core where the business application is located, the business application category, the business application ID, etc. The classification and summarization module 1006 can also be configured to perform traffic statistics on the business application according to the summarization method of multiple cores and multiple services.

[0125] According to one embodiment of this application, the classification and summarization module 1006 includes a data structure for classification statistics. This data structure can be designed according to the classification requirements of business applications. The data structure can be, for example, a hash table. For example, when recording traffic statistics, the application category can be the key value in the hash table, and the application traffic data corresponding to the application category can be the value value.

[0126] According to one embodiment of this application, the classification and summarization module 1006 is configured to read the traffic statistics data of the memory block and perform traffic summary statistics through a hash table according to the category of business requirements. Specifically, the data caching module 1005 can pass the traffic statistics data of the memory block required by the business application in the data caching module 1005 to the classification and summarization module 1006 by calling the interface of the classification and summarization module 1006.

[0127] According to one embodiment of this application, the classification and summarization module 1006 is further configured to write the summarized traffic statistics data into the database 1008. The database 1008 can also provide a query interface for business personnel to query historical summarized traffic.

[0128] According to one embodiment of this application, the classification and summarization module 1006 is further configured to write the summarized traffic statistics data into a shared memory block, provide a data source for business queries of summarized traffic, and provide a query interface for the shared memory block.

[0129] According to one embodiment of this application, when querying the aggregated traffic, the Web management system 1007 can query the aggregated statistical data through the traffic query interface of the classification and summary module 1006, and obtain the aggregated historical traffic data through the database 1008 query interface.

[0130] According to one embodiment of this application, the business application process module 1001 can traverse all memory blocks through the query interface provided by the memory pool module 1003, determine whether the memory block is occupied based on the Flag flag in the management structure, and obtain the IP, traffic statistics and other values ​​of the occupied memory block.

[0131] An electronic device according to one embodiment of the present application includes a processor and a memory, wherein an executable program is stored in the memory, and the memory executes the executable program to perform any of the above-described traffic statistics methods.

[0132] According to one embodiment of the present application, a storage medium carries one or more computer programs, and when the one or more computer programs are executed by a processor, they represent any of the above-mentioned traffic statistics methods.

[0133] Figure 2 A schematic diagram illustrating the structure of a flow statistics device implemented using a processing system is shown. The flow statistics device may include corresponding modules that execute one or more steps in the corresponding flowchart described above. Therefore, each or more steps in the flowchart can be executed by a corresponding module, and the device may include one or more of these modules. Modules may be one or more hardware modules specifically configured to execute the corresponding steps, or implemented by a processor configured to execute the corresponding steps, or stored in a computer-readable medium for processor implementation, or implemented through some combination thereof.

[0134] This hardware architecture can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0135] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0136] The processor 1200 described above can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0137] Any process or method described in the flowcharts or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this application pertain. The processor performs the various methods and processes described above. For example, the method embodiments of this application can be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some embodiments, part or all of the software program may be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).

[0138] The logic and / or steps represented in the flowchart or otherwise described herein may be specifically implemented in any readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0139] For the purposes of this specification, a "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM). Furthermore, a readable storage medium can even be paper or other suitable media on which a program can be printed, since a program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in memory.

[0140] It should be understood that various parts of this application can be implemented in hardware, software, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0141] Those skilled in the art will understand that all or part of the steps of the methods described above can be implemented by a program instructing related hardware, and the program can be stored in a readable storage medium. When executed, the program includes one or a combination of the steps of the method implementation.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0143] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0145] Those skilled in the art should understand that the above embodiments are merely for illustrative purposes and are not intended to limit the scope of this application. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of this application.

Claims

1. A method for traffic flow statistics, characterized in that, include: Initialize the traffic statistics process, which includes at least initializing the memory pool and message transmission based on the first data created in shared memory. Establish business application connections based on business application processes to generate business application data traffic; as well as The traffic statistics process is used to collect statistics on the traffic of the service application during data transmission; wherein: The first data includes a lock-free circular queue array; The initialization of the memory pool includes: creating a first lock-free circular queue array in shared memory, which is used to store the memory block management structure of the memory blocks required by the business application, wherein the index of the first lock-free circular queue array corresponds one-to-one with the CPU core number; The message transmission initialization includes: creating a second lock-free circular queue array in shared memory, which is used to dynamically write the memory block addresses stored in the first lock-free circular queue array, wherein the index of the second lock-free circular queue array corresponds one-to-one with the CPU core number.

2. The flow rate statistics method according to claim 1, characterized in that, The member data of the memory block management structure includes at least the business application address, which is used to identify the address pointer of the memory block required by the business application.

3. The flow rate statistics method according to claim 2, characterized in that, The initialization of the traffic statistics process also includes data cache initialization, which includes: Create a singly linked list, which is used to cache pointer addresses of memory blocks required by the business application, obtained from the second lock-free circular queue array; and Start a timer to periodically traverse the singly linked list and periodically obtain the pointer addresses of the memory blocks required by the business application from the second lock-free circular queue array.

4. The flow rate statistics method according to claim 3, characterized in that, The initialization of the traffic statistics process also includes category and summary initialization, which includes: Create a hash table, which is used to record traffic information generated by multiple different business applications during traffic statistics.

5. The flow rate statistics method according to claim 4, characterized in that, Establishing business application connections based on business application processes includes: Allocate the memory blocks required by the business application; Save the pointer addresses of the memory blocks required by the business application to the business connection; The memory block required by the business application is initialized based on the first business message data of the business application; Write the pointer addresses of the memory blocks required by the business application into the first lock-free circular queue array; and Traverse the second lock-free circular queue array, read the pointer address of the memory block required by the business application, and save it to the singly linked list.

6. The flow rate statistics method according to claim 5, characterized in that, The memory blocks required by the business application are initialized based on the first business message data of the business application, including: Based on the first business message data of the business application, the memory blocks required by the business application are initialized through the memory block management structure; The member data of the memory block management structure also includes the IP address, port, application protocol, traffic statistics value and Flag of the business application. The Flag is used to indicate whether the memory block required by the business application is occupied. The initialization of the memory block required by the business application based on the first business message data of the business application through the memory block management structure includes: obtaining the IP address, port, and application protocol of the business application based on the first business message data of the business application, assigning the IP address, port, and application protocol of the business application to the corresponding member data of the memory block management structure, initializing the traffic statistics value to zero, and initializing the Flag to indicate that the memory block required by the business application is occupied.

7. The flow statistics method according to claim 6, characterized in that, The traffic statistics process involves counting the traffic of the service application during data transmission, including: When the business application process processes data packets after the first business packet, it accesses the memory block management structure through the pointer address of the memory block required by the business application stored in the business connection, and updates the traffic statistics value of the memory block management structure with the business traffic data; The system periodically accesses the singly linked list and reads the data of the business application through the pointer addresses of the memory blocks required by the business application in the singly linked list; and The data from the aforementioned business applications is categorized, recorded using a hash table, and traffic is aggregated and statistically analyzed according to different categories.

8. The flow rate statistics method according to claim 7, characterized in that, It also includes the business application closing the connection, which includes: The member data of the memory block management structure also includes a timestamp marker, which is used to identify the start and end times of the business application traffic statistics; When the process of the business application receives a data packet to close the connection, it accesses the memory block management structure through the pointer address of the memory block required by the business application in the business connection, updates the Flag in the memory block management structure to the end flag, and updates the timestamp flag.

9. The flow rate statistics method according to claim 8, characterized in that, It also includes releasing resources, which includes: Query the second lock-free circular queue, read the pointer address of the memory block required by the business application, and write it into the singly linked list; Traverse the singly linked list and read the data of the memory blocks required by the business application, classify and count the data of the memory blocks required by the business application, write the statistical data in the hash table into the database, and clear the hash table. Remove the pointer addresses of the memory blocks required by the business application from the singly linked list; and Release the memory blocks required by the business applications in the memory pool.

10. A flow rate counting device, characterized in that, include: The initialization module is used to initialize the traffic statistics process, including at least memory pool initialization based on the first data created in shared memory and message transmission initialization. The business application process module is used to establish business application connections based on business application processes to generate business application data traffic; as well as The business traffic statistics module is used to perform traffic statistics on the business application through the traffic statistics process; wherein: The first data includes a lock-free circular queue array; The initialization of the memory pool includes: creating a first lock-free circular queue array in shared memory, which is used to store the memory block management structure of the memory blocks required by the business application, wherein the index of the first lock-free circular queue array corresponds one-to-one with the CPU core number; The message transmission initialization includes: creating a second lock-free circular queue array in shared memory, which is used to dynamically write the memory block addresses stored in the first lock-free circular queue array, wherein the index of the second lock-free circular queue array corresponds one-to-one with the CPU core number.

11. A flow statistics system, characterized in that, include: The business application process module is used to establish business application connections to generate business application data traffic. The traffic statistics process module is used to count the data traffic of the business application based on the initialization traffic statistics process and obtain traffic statistics information. The initialization of the traffic statistics process includes at least the initialization of the memory pool based on the first data created in shared memory and the initialization of message transmission. The management system is used to enable users to query the traffic statistics of the business application in a human-computer interaction manner based on the traffic statistics information. as well as A database is used to store the traffic statistics information; wherein: The first data includes a lock-free circular queue array; The initialization of the memory pool includes: creating a first lock-free circular queue array in shared memory, which is used to store the memory block management structure of the memory blocks required by the business application, wherein the index of the first lock-free circular queue array corresponds one-to-one with the CPU core number; The message transmission initialization includes: creating a second lock-free circular queue array in shared memory, which is used to dynamically write the memory block addresses stored in the first lock-free circular queue array, wherein the index of the second lock-free circular queue array corresponds one-to-one with the CPU core number.

12. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores an executable program, and the memory executes the executable program to perform the traffic statistics method as described in any one of claims 1 to 9.

13. A storage medium, characterized in that, The storage medium carries one or more computer programs, which, when executed by a processor, implement the traffic statistics method according to any one of claims 1 to 9.

Images