Millisecond traffic monitoring method and device, electronic equipment and storage medium
By grabbing and analyzing the network traffic data of the vehicle, the millisecond -level traffic monitoring is achieved, the problem of unable to analyze the instantaneous behavior of traffic in the existing technology, and optimize the data processing and resource occupation.
Patent Information
- Application Number
- CN202411482160.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-01-24
AI Technical Summary
The prior art is difficult to achieve millisecond monitoring of network traffic, resulting in the inability to analyze the instantaneous behavior of traffic in fine-grained and position the reasons for the surge in network traffic.
By grasping the traffic data of the vehicle, save it into a binary file, and start the asynchronous parsing thread, and statistics are obtained by the statistical statistical results of the millisecond traffic in accordance with the preset statistical rules (global traffic statistical rules, global data structure rules, and data computing rules).
The millisecond monitoring of network traffic is realized, the frequency of calculation is reduced, the data storage and processing optimize, the occupation of system resources is reduced, and the consumption of CPU, memory and I/O resources is reduced to the maximum extent.
Smart Images
Figure CN119363435A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of network traffic monitoring, and in particular to a millisecond-level traffic monitoring method, device, electronic device, and storage medium. Background Art
[0002] Among the performance requirements of autonomous driving, low network latency is an important performance indicator, which involves the interaction of network traffic between various sub-module systems. If poor network conditions occur, it may cause more serious problems.
[0003] However, when high-latency networks appear, on-site network monitoring is often not granular enough to the millisecond level. For example, if statistics are taken at the second level, the traffic between the various pre-controls will be relatively normal, and sudden traffic anomalies will not be discovered. As a result, it is impossible to comprehensively analyze the instantaneous behavior of the traffic, and it is impossible to locate the cause of the surge in network traffic. Summary of the invention
[0004] The embodiments of the present application provide a millisecond-level traffic monitoring method, device, electronic device, and storage medium to achieve millisecond-level traffic monitoring, thereby analyzing and controlling traffic conditions.
[0005] The present application embodiment adopts the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a millisecond-level traffic monitoring method, wherein the method includes:
[0007] Capturing traffic data of the vehicle, the traffic data at least including target group information and data packet size;
[0008] Saving the target group information and the data packet size in the traffic data as a binary file in binary format;
[0009] An asynchronous parsing thread is started according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, millisecond-level traffic monitoring results are obtained according to preset statistical rules. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0010] In some embodiments, the global traffic statistics rule includes:
[0011] Set the preset statistical period to perform global traffic statistics every n milliseconds;
[0012] The flow data is read from the binary file, and the flow data is aggregated into a global data structure to obtain statistical results.
[0013] In some embodiments, the global data structure rules include:
[0014] A global map structure is used to store the statistical results;
[0015] The key in the map refers to the target group information, and the value refers to the structure containing traffic statistics data;
[0016] The structure includes at least one of the following indicator information: the number of times the flow exceeds the standard, the flow peak value, and the global flow size corresponding to the peak value,
[0017] The number of traffic exceeding the standard is used as the number of times the traffic of each quintuple exceeds the standard in the past time period;
[0018] The traffic peak value is used as the maximum traffic value of each quintuple in the past time period;
[0019] The global flow size corresponding to the peak value is used as the global flow size when the flow peak value occurs.
[0020] In some embodiments, the method further comprises: updating the millisecond-level flow monitoring result,
[0021] By counting traffic every n milliseconds, the newly generated data is read and the corresponding five-tuple statistics in the global map are updated;
[0022] At each update, the new statistical data is merged with the existing data and the number of times the exceeding standard is reached, the flow peak value and the corresponding global flow size are updated.
[0023] In some embodiments, the method further comprises:
[0024] When obtaining millisecond-level traffic monitoring results, limit the statistical frequency to once every 10 milliseconds;
[0025] and / or,
[0026] In the statistical process of obtaining the millisecond-level traffic monitoring results, only one I / O operation is performed to write the statistical results into the global map.
[0027] In some embodiments, the method further includes a method of simplifying the millisecond-level traffic monitoring result:
[0028] Use fixed statistical windows to control the amount of computation and storage requirements;
[0029] and / or,
[0030] Use peak recording method to record the flow peak;
[0031] and / or,
[0032] The number of times the flow exceeds the standard is recorded, and the abnormal flow situation is evaluated.
[0033] In some embodiments, the method further comprises the step of optimizing the performance of the millisecond level traffic monitoring results:
[0034] Multithreaded optimization technology is used in statistical operations;
[0035] and / or,
[0036] Use caching technology to cache commonly used statistics in memory.
[0037] and / or,
[0038] In combination with the process ID and process name provided by the middleware, the target group information is matched with the relevant process.
[0039] In a second aspect, an embodiment of the present application further provides a millisecond-level flow monitoring device, wherein the device comprises:
[0040] A capture module, used to capture the traffic data of the vehicle, wherein the traffic data at least includes target group information and data packet size;
[0041] A saving module, used to save the target group information and data packet size in the traffic data as a binary file in binary format;
[0042] A monitoring module is used to start an asynchronous parsing thread according to the binary file, and obtain millisecond-level traffic monitoring results according to preset statistical rules when it is determined that the vehicle is in a non-automatic driving state. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0043] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.
[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.
[0045] At least one of the above technical solutions adopted in the embodiment of the present application can achieve the following beneficial effects: capturing the traffic data of the vehicle, and the traffic data at least includes the target group information and the data packet size. The target group information and the data packet size in the traffic data are saved as a binary file in binary format, thereby completing the traffic capture. Then, the asynchronous parsing thread is started according to the binary file, and the millisecond-level traffic monitoring results are obtained according to the preset statistical rules when it is judged that the vehicle is in a non-automatic driving state. The preset statistical rules for the analysis and processing stage of the millisecond-level traffic at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to the preset statistical period and the global data structure rules save the statistical results according to the global mapping structure. Through the above method, the calculation frequency is reduced, the data storage and processing are optimized, and the occupation of system resources is reduced. And through millisecond global statistics, streamlined data structure and efficient calculation rules. In addition, while ensuring statistical accuracy, the above method minimizes the consumption of CPU, memory and I / O resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0047] Figure 1 A flow chart of a millisecond-level flow monitoring method in an embodiment of the present application;
[0048] Figure 2 A schematic diagram of the implementation principle of the millisecond-level flow monitoring method in an embodiment of the present application;
[0049] Figure 3 This is a schematic diagram of the structure of a millisecond-level flow monitoring device in an embodiment of the present application;
[0050] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0052] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0053] The present application embodiment provides a millisecond level flow monitoring method, such as Figure 1 As shown, a flow chart of a method for monitoring millisecond-level traffic flow in an embodiment of the present application is provided, and the method at least includes the following steps S110 to S130:
[0054] Step S110, capturing vehicle traffic data, wherein the traffic data at least includes target group information and data packet size.
[0055] The traffic data of vehicles mainly comes from the traffic of vehicle domain controllers. The domain controller (Domain Controller Unit, DCU) integrates multiple electronic control units (Electronic Control Unit, ECU) scattered in traditional cars into one or several powerful processors to achieve centralized control of specific functional domains. The domain controller mainly includes the domain master processor (Domain Master Controller), operating system (Operating System), application software and algorithms. The network traffic of the domain controller directly affects the performance of the vehicle in achieving autonomous driving.
[0056] Optionally, during the process of capturing traffic data, the traffic passing through the network card can be captured based on the libcap library.
[0057] Step S120, saving the target group information and data packet size in the traffic data into a binary file in binary format.
[0058] In specific implementation, during the capture process, only the target group information and the data packet size are retained, and this information is saved in a file in binary format. This approach significantly reduces the storage space occupied.
[0059] It can be understood that the target group information can be a quintuple information, including source IP, source port, destination IP, destination port, and protocol. The quintuple is a communication term, usually referring to the source IP address, source port, destination IP address, destination port, and transport layer protocol.
[0060] Step S130, starting an asynchronous parsing thread according to the binary file, and obtaining millisecond-level traffic monitoring results according to preset statistical rules when it is determined that the vehicle is in a non-automatic driving state. The preset statistical rules include at least global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0061] After the thread for capturing traffic data is started, an independent parsing thread is started to process the newly generated files to be parsed. The parsing thread will check whether it is in the automatic driving state. If so, the thread will temporarily give up the CPU resources and enter the dormant state. The parsing thread will only start to parse the data when it is not in automatic driving. At the same time, in order to meet the needs of millisecond-level traffic statistics, frequent calculations will consume a lot of system resources. Therefore, the statistical rules have been streamlined to balance performance and resource requirements.
[0062] The global traffic statistics rule performs global traffic statistics according to a preset statistics period, that is, the statistics period is set to perform global traffic statistics once every fixed milliseconds.
[0063] The global data structure rule saves the statistical results according to the global mapping structure, that is, a global map structure is used to save the statistical results.
[0064] Preferably, the millisecond-level traffic monitoring results obtained by statistics are uploaded to the cloud, and the above method is executed on the vehicle side.
[0065] Through the above method, streamlined data capture and statistical rules are adopted to achieve accurate monitoring and analysis of instantaneous traffic while ensuring low resource usage. Furthermore, the traffic capture and analysis process is optimized to reduce interference with system resources. Finally, through the precise positioning of the process, traffic optimization can be effectively performed to improve the overall performance and stability of the system.
[0066] Different from the related technologies, most of which are aimed at second-level or minute-level traffic monitoring, the above method starts an asynchronous parsing thread according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, the millisecond-level traffic monitoring results of the domain controller are obtained according to the preset statistical rules, thereby realizing millisecond-level traffic monitoring.
[0067] Different from the related technologies, if the statistics are based on the second level, the sudden abnormal traffic situation cannot be found, resulting in the inability to comprehensively analyze the instantaneous behavior of the traffic, and it is impossible to locate the cause of the surge in network traffic. Through the above method, the precise positioning of the process can effectively optimize the traffic and improve the overall performance and stability of the system.
[0068] In one embodiment of the present application, the global traffic statistics rule includes: setting a preset statistical period to perform global traffic statistics every n milliseconds; reading traffic data from the binary file, and aggregating the traffic data into a global data structure to obtain statistical results.
[0069] Global traffic statistics, the statistical cycle is set to perform global traffic statistics every n milliseconds. The operations within the statistical cycle mainly include reading traffic data from the captured binary file and aggregating the traffic data into the global data structure. The use of frequent statistical frequencies can provide high-resolution data to capture instantaneous traffic changes. Among them, n is the configuration result, and technicians in this field can configure it according to actual usage.
[0070] In one embodiment of the present application, the global data structure rules include: using a global map mapping structure to save the statistical results; the key in the map refers to the target group information, and the value refers to a structure containing traffic statistical data; the structure includes at least one of the following indicator information: the number of traffic excesses, the traffic peak, and the global traffic size corresponding to the peak, the number of traffic excesses is used as the number of times the traffic of each quintuple exceeds the standard in the past time period; the traffic peak is used as the maximum traffic value of each quintuple in the past time period; the global traffic size corresponding to the peak is used as the global traffic size when the traffic peak occurs.
[0071] The global data structure uses a global map structure to store the statistics. The key value in the map is the target group information (source IP, source port, destination IP, destination port, protocol), and the value value is a structure containing traffic statistics.
[0072] The structure records the global traffic at the peak traffic volume:
[0073] (1) Number of traffic exceeding the limit: refers to the number of times the traffic of each quintuple exceeds the limit in the past minute.
[0074] (2) Traffic peak: refers to the maximum traffic value of each quintuple in the past minute.
[0075] (3) Global flow size corresponding to the peak value: refers to the global flow size when the flow peak occurs.
[0076] The statistical rules provided in the embodiments of the present application are adopted to reduce CPU consumption and IO resources. The performance requirements of the machine are not high and it can be quickly deployed, simple and easy to operate. The output json file is simple and clear, and the problem can be quickly located.
[0077] In one embodiment of the present application, the method also includes: updating the millisecond-level traffic monitoring results, by counting the traffic once every n milliseconds, reading the newly generated data and updating the statistical information of the corresponding five-tuple in the global map; at each update, merging the new statistical data with the existing data and updating the number of excess times, the traffic peak and the corresponding global traffic size.
[0078] When updating statistics, traffic is counted every n milliseconds, new data is read, and the corresponding five-tuple statistics in the global map are updated. At each update, the new statistics are simply merged with the existing data, and the number of exceedances, traffic peaks, and the corresponding global traffic size are updated.
[0079] Preferably, outdated data needs to be deleted: In order to control the size of the map and improve performance, outdated data can be cleaned up regularly, for example, deleting historical data older than 1 minute.
[0080] In one embodiment of the present application, the method also includes: when obtaining millisecond-level traffic monitoring results, limiting the statistical frequency to once every 10 milliseconds; and / or, performing only one I / O operation during the statistical process of obtaining millisecond-level traffic monitoring results, and writing the statistical results into the global map.
[0081] Since traffic statistics involve a lot of data calculations, if 1,000 calculations are performed per second, it will greatly occupy system resources. In order to reduce the calculation load, the statistics frequency is limited to once every 10 milliseconds. Reduce the calculation frequency. This makes global traffic statistics 100 times per second, reducing the calculation frequency and reducing the pressure on the CPU.
[0082] By performing only one I / O operation during the statistical process, that is, writing the statistical results to the global map, this method reduces frequent disk I / O operations, further reduces system resource consumption, and thus reduces I / O operations.
[0083] It can be understood that any one or more of the above calculation methods can be selected, and are not specifically limited in the embodiments of the present application.
[0084] In one embodiment of the present application, the method also includes a method for streamlining the millisecond-level traffic monitoring results: using a fixed statistical window to control the amount of calculation and storage requirements; and / or, using a peak recording method to record traffic peaks; and / or, using an excess number recording method to record the number of traffic excesses and evaluate abnormal traffic conditions.
[0085] A simplified processing flow for statistical data, in which simplified calculation rules are adopted: In order to reduce the use of computing resources, the simplified rules for statistical data include: fixed statistical window, peak value record, and number of times exceeding the standard are recorded.
[0086] Specifically, a fixed statistical window refers to setting a 1-minute statistical window to control the amount of calculation and storage requirements. Peak recording refers to recording only the peak flow rate, rather than data at all time points, to reduce data storage and computing burden. Exceeding times recording refers to recording the number of times the flow exceeds the standard, which is used to evaluate abnormal flow conditions, rather than recording the specific data of each exceeding standard in detail.
[0087] It should be noted that the above-mentioned fixed statistical window, peak value record, and number of exceeding limit times record can be used simultaneously or any one or more of them can be selected for use in combination.
[0088] It can be understood that any one or more of the above optimization methods can be selected, and are not specifically limited in the embodiments of the present application.
[0089] In one embodiment of the present application, the method also includes a step of optimizing the performance of millisecond-level traffic monitoring results: using multi-threaded optimization technology in statistical operations; and / or using caching technology to cache commonly used statistical data in memory, and / or, combining the process ID and process name provided by the middleware, to correspond the target group information to the relevant process.
[0090] Optimizing the performance of millisecond-level traffic monitoring results includes but is not limited to multi-threading optimization, data caching, and process positioning and optimization.
[0091] (1) Multithreading optimization: Using multithreading technology in statistical operations can further improve performance. For example, traffic data can be read and merged in one thread, and statistical results can be calculated and updated in another thread. Multithreading optimization can fully utilize the processing power of multi-core CPUs and improve the efficiency of the overall system.
[0092] (2) Data caching: Use caching technology to cache commonly used statistical data in memory, reducing the frequency of access to the global map. This can speed up data reading and improve statistical efficiency.
[0093] (3) Process location and optimization: In order to accurately find the processes with high traffic, the target group information is matched with the specific process by combining the process ID (PID) and process name provided by the middleware. Through this correspondence, the process that generates high traffic can be quickly located, so as to perform targeted traffic optimization and adjustment.
[0094] The rule of filtering statistical results by thresholds can filter out a large amount of useless statistical information.
[0095] It can be understood that any one or more of the above optimization methods can be selected, and are not specifically limited in the embodiments of the present application.
[0096] like Figure 2 As shown, it is a schematic diagram of the implementation principle of the millisecond-level flow monitoring method in the embodiment of the present application, taking the global flow statistics as an example once every 10 milliseconds, specifically including the following steps:
[0097] Step S1, receiving the package data packet from the vehicle domain controller, and relying on the libcap library to capture the traffic passing through the network card.
[0098] Step S2, parse the header and save it into a binary file.
[0099] Step S3, obtain the binary file that has been parsed, asynchronous parsing thread (only runs in non-autonomous driving state)
[0100] Step S4, parse for sub in packages one by one to get header_info.
[0101] Step S5, determine whether the timestamp of the package is within 10ms. If not, proceed to step S10.
[0102] Step S6, if yes, map<header_info,size> Check if the header exists. If it does, updatesize will update the data. If it does not exist, insert will insert new data.
[0103] Step S7: If not, sort the quintuples in the map and output the statistics, and then clear the map.
[0104] Step S8, filter the data whose size is greater than the threshold, and record the global flow and the flow size of the top 10 quintuples within 10 ms.
[0105] Step S9, store in global map<header_info,info> , where info stores the number of times the header exceeds the threshold, the highest value so far, and the corresponding global traffic.
[0106] Step S10, if timstamp>=1min, perform statistical sorting on the global map and then output json to a file.
[0107] Step S11, upload to the cloud, can visualize and count the traffic changes.
[0108] The present application embodiment also provides a millisecond level flow monitoring device 300, such as Figure 3 As shown, a schematic diagram of the structure of a millisecond-level flow monitoring device in an embodiment of the present application is provided, wherein the millisecond-level flow monitoring device 300 at least includes: a capture module 310, a storage module 320, and a monitoring module 330, wherein:
[0109] In one embodiment of the present application, the capture module 310 is specifically used to capture vehicle traffic data, where the traffic data at least includes target group information and data packet size.
[0110] The traffic data of vehicles mainly comes from the traffic of vehicle domain controllers. The domain controller (Domain Controller Unit, DCU) integrates multiple electronic control units (Electronic Control Unit, ECU) scattered in traditional cars into one or several powerful processors to achieve centralized control of specific functional domains. The domain controller mainly includes the domain master processor (Domain Master Controller), operating system (Operating System), application software and algorithms. The network traffic of the domain controller directly affects the performance of the vehicle in achieving autonomous driving.
[0111] Optionally, during the process of capturing traffic data, the traffic passing through the network card can be captured based on the libcap library.
[0112] In one embodiment of the present application, the capture and save module 320 is specifically used to save the target group information and the data packet size in the traffic data as a binary file in a binary format.
[0113] In specific implementation, during the capture process, only the target group information and the data packet size are retained, and this information is saved in a file in binary format. This approach significantly reduces the storage space occupied.
[0114] It can be understood that the target group information includes source IP, source port, destination IP, destination port, and protocol. The quintuple is a communication term, usually referring to the source IP address, source port, destination IP address, destination port, and transport layer protocol.
[0115] In one embodiment of the present application, the monitoring module 330 is specifically used to: start an asynchronous parsing thread according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, obtain millisecond-level traffic monitoring results according to preset statistical rules. The preset statistical rules include at least global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0116] After the thread for capturing traffic data is started, an independent parsing thread is started to process the newly generated files to be parsed. The parsing thread will check whether it is in the automatic driving state. If so, the thread will temporarily give up the CPU resources and enter the dormant state. The parsing thread will only start to parse the data when it is not in automatic driving. At the same time, in order to meet the needs of millisecond-level traffic statistics, frequent calculations will consume a lot of system resources. Therefore, the statistical rules have been streamlined to balance performance and resource requirements.
[0117] The global traffic statistics rule performs global traffic statistics according to a preset statistics period, that is, the statistics period is set to perform global traffic statistics once every fixed milliseconds.
[0118] The global data structure rule saves the statistical results according to the global mapping structure, that is, a global map structure is used to save the statistical results.
[0119] In one embodiment of the present application, the monitoring module 310 is also used to
[0120] Set the preset statistical period to perform global traffic statistics every n milliseconds;
[0121] The flow data is read from the binary file, and the flow data is aggregated into a global data structure to obtain statistical results.
[0122] In one embodiment of the present application, the monitoring module 310 is also used to
[0123] A global map structure is used to store the statistical results;
[0124] The key in the map refers to the target group information, and the value refers to the structure containing traffic statistics data;
[0125] The structure includes at least one of the following indicator information: the number of times the flow exceeds the standard, the flow peak value, and the global flow size corresponding to the peak value,
[0126] The number of traffic exceeding the standard is used as the number of times the traffic of each quintuple exceeds the standard in the past time period;
[0127] The traffic peak value is used as the maximum traffic value of each quintuple in the past time period;
[0128] The global flow size corresponding to the peak value is used as the global flow size when the flow peak value occurs.
[0129] In one embodiment of the present application, it also includes updating the millisecond level flow monitoring result module for
[0130] By counting traffic every n milliseconds, the newly generated data is read and the corresponding five-tuple statistics in the global map are updated;
[0131] At each update, the new statistical data is merged with the existing data and the number of times the exceeding standard is reached, the flow peak value and the corresponding global flow size are updated.
[0132] In one embodiment of the present application, it further includes: a load calculation module for
[0133] When obtaining millisecond-level traffic monitoring results, limit the statistical frequency to once every 10 milliseconds;
[0134] and / or,
[0135] In the statistical process of obtaining the millisecond-level traffic monitoring results, only one I / O operation is performed to write the statistical results into the global map.
[0136] In one embodiment of the present application, it also includes a module for streamlining the millisecond-level flow monitoring result, for
[0137] Use fixed statistical windows to control the amount of computation and storage requirements;
[0138] and / or,
[0139] Use peak recording method to record the flow peak;
[0140] and / or,
[0141] The number of times the flow exceeds the standard is recorded, and the abnormal flow situation is evaluated.
[0142] In one embodiment of the present application, a performance module for optimizing millisecond-level flow monitoring results is also included.
[0143] Multithreaded optimization technology is used in statistical operations;
[0144] and / or,
[0145] Use caching technology to cache commonly used statistics in memory.
[0146] and / or,
[0147] In combination with the process ID and process name provided by the middleware, the target group information is matched with the relevant process.
[0148] It can be understood that the above-mentioned millisecond-level traffic monitoring device can implement the various steps of the millisecond-level traffic monitoring method provided in the aforementioned embodiment, and the relevant explanations about the millisecond-level traffic monitoring method are applicable to the millisecond-level traffic monitoring device and will not be repeated here.
[0149] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0150] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0151] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0152] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a millisecond-level flow monitoring device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0153] Capturing traffic data of the vehicle, the traffic data at least including target group information and data packet size;
[0154] Saving the target group information and the data packet size in the traffic data as a binary file in binary format;
[0155] An asynchronous parsing thread is started according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, millisecond-level traffic monitoring results are obtained according to preset statistical rules. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0156] The above application Figure 1The method performed by the millisecond-level flow monitoring device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0157] The electronic device may also perform Figure 1 The method for executing the millisecond level flow monitoring device in the Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.
[0158] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the millisecond level flow monitoring device in the embodiment shown is specifically used to perform:
[0159] Capturing traffic data of the vehicle, the traffic data at least including target group information and data packet size;
[0160] Saving the target group information and the data packet size in the traffic data as a binary file in binary format;
[0161] An asynchronous parsing thread is started according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, millisecond-level traffic monitoring results are obtained according to preset statistical rules. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
[0162] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0163] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0164] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0166] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0167] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0168] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0169] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0171] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A millisecond-level flow monitoring method, wherein: The method comprises: Capturing traffic data of the vehicle, the traffic data at least including target group information and data packet size; Saving the target group information and the data packet size in the traffic data as a binary file in binary format; An asynchronous parsing thread is started according to the binary file, and when it is determined that the vehicle is in a non-automatic driving state, millisecond-level traffic monitoring results are obtained according to preset statistical rules. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
2. The method of claim 1, wherein: The global traffic statistics rule includes: Set the preset statistical period to perform global traffic statistics every n milliseconds; The flow data is read from the binary file, and the flow data is aggregated into a global data structure to obtain statistical results.
3. The method of claim 2, wherein: The global data structure rules include: A global map structure is used to store the statistical results; The key in the map refers to the target group information, and the value refers to the structure containing traffic statistics data; The structure includes at least one of the following indicator information: the number of times the flow exceeds the standard, the flow peak value, and the global flow size corresponding to the peak value, The number of traffic exceeding the standard is used as the number of times the traffic of each quintuple exceeds the standard in the past time period; The traffic peak value is used as the maximum traffic value of each quintuple in the past time period; The global flow size corresponding to the peak value is used as the global flow size when the flow peak value occurs.
4. The method of claim 2, wherein: The method further comprises: updating the millisecond-level flow monitoring result, By counting traffic every n milliseconds, the newly generated data is read and the corresponding five-tuple statistics in the global map are updated; At each update, the new statistical data is merged with the existing data and the number of times the exceeding standard is reached, the flow peak value and the corresponding global flow size are updated.
5. The method of claim 3, wherein: The method further comprises: When obtaining millisecond-level traffic monitoring results, limit the statistical frequency to once every 10 milliseconds; and / or, In the statistical process of obtaining the millisecond-level traffic monitoring results, only one I / O operation is performed to write the statistical results into the global map.
6. The method of claim 1, wherein: The method also includes a method for simplifying the millisecond-level flow monitoring results: Use fixed statistical windows to control the amount of computation and storage requirements; and / or, Use peak recording method to record the flow peak; and / or, The number of times the flow exceeds the standard is recorded, and the abnormal flow situation is evaluated.
7. The method of claim 1, wherein: The method further comprises the step of optimizing the performance of the millisecond level flow monitoring results: Multithreaded optimization technology is used in statistical operations; and / or, Use caching technology to cache frequently used statistics in memory; and / or, In combination with the process ID and process name provided by the middleware, the target group information is matched with the relevant process.
8. A millisecond-level flow monitoring device, wherein: The device comprises: A capture module, used to capture the traffic data of the vehicle, wherein the traffic data at least includes target group information and data packet size; A saving module, used to save the target group information and data packet size in the traffic data as a binary file in binary format; The monitoring module is used to start an asynchronous parsing thread according to the binary file, and obtain millisecond-level traffic monitoring results according to preset statistical rules when it is determined that the vehicle is in a non-automatic driving state. The preset statistical rules at least include global traffic statistical rules, global data structure rules, and data calculation rules. The global traffic statistical rules perform global traffic statistics according to a preset statistical period, and the global data structure rules save statistical results according to a global mapping structure.
9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 7.
10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 7.