Data capturing and analyzing method, system and device for charging pile port monitoring

Abstract

The invention discloses a data capturing and analyzing method, system and device for charging pile port monitoring, and the method comprises the steps: building a packet capturing platform, inputting a to-be-captured port through a configuration file, capturing a TCP data packet of the to-be-captured port through the packet capturing platform, and analyzing the TCP data packet to obtain analyzed data; and sending the analyzed data to a message queue through RabbitMQ, and distributing the analyzed data to a specified router. According to the main technical scheme and effects, the package capturing platform adopts a multi-thread design, so that the smooth running of the process is ensured, the jamming caused by a long-time task is avoided, and the data capturing and analyzing efficiency is greatly improved; the real-time captured TCP data is broadcasted by adopting a RabbitMQ message queue, so that distributed storage and remote sharing of the TCP data are realized, and the efficiency of network operation and maintenance and fault analysis and solution of developers is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data capture and analysis, and in particular to a data capture and analysis method, system and device for monitoring a charging pile port. Background Art

[0002] With the increasing popularity of new energy vehicles, the number of charging piles, as supporting infrastructure, continues to increase. Intelligent management and remote monitoring of charging piles have become key to the industry's development. Demand for remote monitoring and data collection of charging piles is growing, including real-time monitoring of charging pile status, charging progress, fault alarms, and data analysis.

[0003] Currently, network packet capture is subject to various geographical restrictions and security flaws. For example, if a charging station device located abroad experiences malfunction, it is necessary to investigate whether the problem lies with the charging station server or the device. However, the commonly used packet capture tool, Wireshark, is a single-threaded network analysis tool that encounters performance bottlenecks when processing large numbers of data packets simultaneously. This is especially true in high-load network environments, resulting in slower packet capture speeds or excessive resource consumption. Furthermore, Wireshark cannot be shared remotely, making it more difficult for network operators and developers to analyze and resolve faults. Furthermore, its complex operating system further increases the cost of use. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, a multi-threaded and real-time sharing technical solution is needed to realize data capture and analysis for charging pile fault troubleshooting. The present invention discloses a data capture and analysis method, system and device for charging pile fault troubleshooting.

[0005] A data capture and analysis method for charging pile port monitoring includes the following steps:

[0006] Building a packet capture platform, wherein the packet capture platform has a unified user interface;

[0007] Input the port to be captured through the configuration file, and select the channel through the user interface;

[0008] The packet capture platform captures the TCP data packet of the port to be captured, and parses the TCP data packet to obtain parsed data;

[0009] Send the parsed data to the message queue via RabbitMQ and distribute it to the designated router;

[0010] Display the captured TCP data packets and export them as Excel files.

[0011] As an implementable method, the construction of the packet capture platform includes the following steps:

[0012] Use Python+Web technology to build a packet capture platform that integrates channel selection, data capture, data display, and data export modules;

[0013] Use the Tkinter library to build a unified user interface.

[0014] As an implementable method, the method of inputting the port to be captured through the configuration file and selecting the channel through the user interface includes the following steps:

[0015] Input the ports to be captured through the configuration file, where the ports to be captured include the charging pile port and the target server port;

[0016] Select a channel name through the pull-down menu on the user interface;

[0017] Connect to the corresponding RabbitMQ message queue through pika technology according to the channel name.

[0018] As an implementable method, the packet capture platform captures the TCP data packet of the port to be captured, including the following steps:

[0019] The packet capture platform creates a TCP server through the socket library, and the TCP server monitors the port to be captured and supports multi-threaded processing and concurrent port data capture;

[0020] The TCP server starts a thread to run the sniff method and captures TCP data packets through the sniff method.

[0021] As an implementable method, parsing the TCP data packet to obtain parsed data includes the following steps:

[0022] Use the scapy library to parse TCP packets and extract key information, including source IP, destination IP, source port, destination port, TCP flags, and message content;

[0023] Check the TCP flag bit by bit to determine the type of TCP data packet;

[0024] If the TCP data packet contains a message, the message content is recorded in hexadecimal format to obtain parsed data.

[0025] As an implementation method, the parsed data is sent to a message queue via RabbitMQ and distributed to a designated router, including the following steps:

[0026] After parsing, the data is interacted with RabbitMQ using the pika library, automatically detecting the RabbitMQ connection status and connection parameters. If the connection is interrupted, reconnection is attempted. The RabbitMQ connection parameters include the host address, port, user name, password, and virtual host.

[0027] Create a persistent topic switch of topic type;

[0028] Distribute the parsed data to the specified router in the form of a comma-separated string.

[0029] As an implementable method, the packet capture platform provides flexible configuration options, including: the port to be captured, the target IP and target port, the connection parameters of RabbitMQ, and the designated router.

[0030] As an implementation method, the packet capture platform has two built-in TCP data storage methods:

[0031] One is to implement local TCP data storage through logging.handlers.RotatingFileHandler;

[0032] The other is to send the captured TCP data packets to a remote TCP storage server through the RabbitMQ message queue.

[0033] A data capture and analysis system for charging pile port monitoring, including: a packet capture platform building module, a port confirmation module to be captured, a data capture and analysis module, a data distribution module, and a data display and export module.

[0034] A packet capture platform building module is used to build a packet capture platform with a unified user interface;

[0035] A port configuration module for capturing, used to input the port to be captured through a configuration file and select a channel through the user interface;

[0036] A data capture and parsing module, configured to capture the TCP data packets of the port to be captured using the packet capture platform, and parse the TCP data packets to obtain parsed data;

[0037] The data distribution module is used to send the parsed data to the message queue through RabbitMQ and distribute it to the designated router;

[0038] The data display and export module is used to display the captured TCP data packets and export them as Excel files.

[0039] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the following method:

[0040] Building a packet capture platform, wherein the packet capture platform has a unified user interface;

[0041] Input the port to be captured through the configuration file, and select the channel through the user interface;

[0042] The packet capture platform captures the TCP data packet of the port to be captured, and parses the TCP data packet to obtain parsed data;

[0043] Send the parsed data to the message queue via RabbitMQ and distribute it to the designated router;

[0044] Display the captured TCP data packets and export them as Excel files.

[0045] A data capture and analysis device for charging pile port monitoring includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the following method is implemented:

[0046] Building a packet capture platform, wherein the packet capture platform has a unified user interface;

[0047] Input the port to be captured through the configuration file, and select the channel through the user interface;

[0048] The packet capture platform captures the TCP data packet of the port to be captured, and parses the TCP data packet to obtain parsed data;

[0049] Send the parsed data to the message queue via RabbitMQ and distribute it to the designated router;

[0050] Display the captured TCP data packets and export them as Excel files.

[0051] The present invention has significant technical effects due to the adoption of the above technical solutions:

[0052] 1. The packet capture platform of the present invention adopts a multi-threaded design to ensure smooth process operation, avoid stalls caused by long tasks, and greatly improve the efficiency of data capture and analysis;

[0053] 2. The packet capture platform of the present invention uses RabbitMQ message queues to broadcast real-time captured TCP data, realizing distributed storage and remote sharing of TCP data, greatly improving the efficiency of network operation and maintenance and developer fault analysis and resolution;

[0054] 3. The packet capture platform's exception capture and retry mechanism ensures stable and robust operation in complex network environments. All exception information is recorded in detail, facilitating subsequent troubleshooting and ensuring the traceability of exception issues.

[0055] 4. The packet capture platform described in this embodiment provides flexible configuration options, and users can quickly adapt to different usage scenarios by modifying the configuration file;

[0056] 5. The user interface of the packet capture platform described in this embodiment is simple and easy to operate, which greatly reduces the cost of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 1 is a flow chart of the data capture and analysis method for charging pile port monitoring according to the present invention;

[0058] Figure 2 It is an overall schematic diagram of the data capture and analysis system for charging pile port monitoring according to the present invention;

[0059] Figure 3 It is a unified user interface of the packet capture platform described in the embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are intended to explain the present invention but the present invention is not limited to the following embodiments.

[0061] Example 1:

[0062] A data capture and analysis method for charging pile port monitoring, such as Figure 1 As shown, the following steps are included:

[0063] S100: Building a packet capture platform, wherein the packet capture platform has a unified user interface;

[0064] S200, input the port to be captured through the configuration file, and select the channel through the user interface;

[0065] S300, the packet capture platform captures the TCP (Transmission Control Protocol) data packet of the port to be captured, and parses the TCP data packet to obtain parsed data;

[0066] S400: Send the parsed data to the message queue via RabbitMQ and distribute it to the designated router;

[0067] S500: Display the captured TCP data packets and export them as an Excel file.

[0068] The TCP server created by the packet capture platform of the present invention adopts a multi-threaded design to ensure smooth process operation, avoid jamming caused by long-term tasks, and greatly improve the efficiency of data capture and analysis; the RabbitMQ message queue is used to broadcast the real-time captured TCP data, realizing the distributed storage and remote sharing of TCP data, greatly improving the efficiency of network operation and maintenance and developer fault analysis and resolution; the exception capture and retry mechanism is adopted to ensure that the packet capture platform runs stably and robustly in a complex network environment, and all exception information is recorded in detail, which is convenient for subsequent problem investigation and ensures the traceability of exception problems; flexible configuration options are provided, and users can quickly adapt to different usage scenarios by modifying the configuration file.

[0069] In this embodiment, building a packet capture platform in step S100 specifically includes the following steps:

[0070] S110: Use Python+Web technology to build a packet capture platform, which integrates channel selection, data capture, data display and data export modules.

[0071] S120: Use the tkinter library (a module in the Python standard library for creating graphical user interfaces) to build a unified user interface (GUI).

[0072] In this embodiment, step S200 inputs the port to be captured through the configuration file and selects the channel through the user interface, which specifically includes the following steps:

[0073] S210: Input ports to be captured through a configuration file, where the ports to be captured include a charging pile port and a target server port.

[0074] S220: Select a channel name through the drop-down menu on the user interface. If the queue already has a consumer, the user is prompted to change the channel.

[0075] S230: Connect to the corresponding RabbitMQ through Pika technology (an artificial intelligence video generation technology) according to the channel name.

[0076] In this embodiment, the packet capture platform in step S300 captures the TCP data packet of the port to be captured and parses the TCP data packet to obtain parsed data, which specifically includes the following steps:

[0077] S310: The packet capture platform creates a TCP server through a socket library (sockets provide an interface for network communication), and the TCP server monitors the port to be captured. The TCP server also forwards the TCP data packet to the target IP and target port.

[0078] S320: The TCP server starts a thread to run the sniff method and captures TCP data packets using the sniff method. The TCP server establishes a connection with the target server through socket.create_connection (a function in the Python standard library socket module that is used to create a TCP connection to a specified address). Two threads are used to transmit data from the charging pile client to the target server and from the target server to the charging pile client. The TCP server supports multi-threaded processing and concurrent port data capture to ensure smooth operation of the user interface and avoid interface freezes caused by long tasks.

[0079] S330: Use the parsing function of the scapy library (an open source Python library for capturing, parsing, generating, and manipulating network data packets) to parse the TCP data packet and extract key information, including the source IP address, destination IP address, source port, destination port, TCP flags, and message content.

[0080] S340: Checks the TCP flag bits bit by bit to determine the type of TCP packet. If the TCP packet contains a message, the message content is recorded in hexadecimal format and truncated to retain only the first 1024 bytes to prevent the leakage of sensitive information. The parsed data is then obtained. Any anomalies that may arise during the parsing process are fully captured to ensure the stable operation of the packet capture platform.

[0081] In this embodiment, step S400 sends the parsed data to a message queue via RabbitMQ and distributes it to a designated router, including the following steps:

[0082] S410: The parsed data uses the pika library (RabbitMQ client library for Python) to interact with RabbitMQ, automatically detect the RabbitMQ connection status and connection parameters, and try to reconnect if the connection is interrupted; the RabbitMQ connection parameters include host address, port, user name, password and virtual host.

[0083] S420: Create a persistent topic switch of topic type;

[0084] S430: Distribute the parsed data as a comma-delimited string to the designated router. After the parsed data is broadcast via the RabbitMQ message queue, users can remotely capture packets through the packet capture platform without having to be present at the debugging site. This allows multiple users at multiple locations to view the parsed data in real time (this packet capture platform supports multiple users capturing the same port simultaneously from multiple locations, resulting in the same results).

[0085] The packet capture platform has two built-in TCP data storage methods:

[0086] One is to implement local TCP data storage through logging.handlers.RotatingFileHandler (a data handler class in the Python standard library logging module), which supports rotation of TCP data packets (maximum 100MB, retaining 5 backups).

[0087] The other is to send the captured TCP data to a remote server through the RabbitMQ message queue, supporting distributed log storage and analysis.

[0088] In this embodiment, step S500 is performed as follows Figure 3 The unified user interface of the packet capture platform shown displays the captured TCP packets and exports them as Excel files.

[0089] S510: Use openpyxl (a Python library for processing Excel files) to export the parsed data as an Excel file.

[0090] S520: Data is displayed in table format and supports clearing and copying functions.

[0091] In this embodiment, to handle exceptions in real time, captured exceptions are recorded in a local log and, as needed, sent to the RabbitMQ message queue. These exceptions include: TCP connection establishment failure, packet parsing anomalies, RabbitMQ connection interruption or message sending failure, and exceptions during data transmission (such as a disconnect). If an exception occurs, a pop-up error message box displays the specific error information. This exception capture and retry mechanism ensures the stable and robust operation of the packet capture platform in complex network environments. All exception information is recorded in detail, facilitating subsequent troubleshooting and ensuring traceability of the exception.

[0092] In addition, the packet capture platform in this embodiment provides flexible configuration options, including: the port to be captured, the target IP and port, the RabbitMQ connection parameters, and the designated router. Users can quickly adapt to different usage scenarios by modifying the configuration file.

[0093] This embodiment provides Figure 3 The simple and user-friendly interface shown provides intuitive operation buttons and error prompts, making it easy for users to learn and operate, further reducing the cost of use.

[0094] The packet capture platform of the present invention adopts a multi-threaded design to ensure smooth process operation, avoid jamming caused by long-term tasks, and greatly improve the efficiency of data capture and analysis; it adopts RabbitMQ message queue to broadcast real-time captured TCP data, realizes distributed storage and remote sharing of TCP data, and greatly improves the efficiency of network operation and maintenance and developer fault analysis and resolution; it adopts exception capture and retry mechanism to ensure stable and robust operation of the packet capture platform in complex network environments, and all exception information is recorded in detail, which is convenient for subsequent problem investigation and ensures the traceability of exception problems; it provides flexible configuration options, and users can quickly adapt to different usage scenarios by modifying the configuration file.

[0095] Example 2:

[0096] A data capture and analysis system for charging pile port monitoring, such as Figure 2 As shown, it includes: packet capture platform building module, to-be-captured port confirmation module, data capture and analysis module, data distribution module, data display and export module,

[0097] A packet capture platform building module is used to build a packet capture platform with a unified user interface;

[0098] A port configuration module for capturing, used to input the port to be captured through a configuration file and select a channel through the user interface;

[0099] A data capture and parsing module, configured to capture the TCP data packets of the port to be captured using the packet capture platform, and parse the TCP data packets to obtain parsed data;

[0100] The data distribution module is used to send the parsed data to the message queue through RabbitMQ and distribute it to the designated router;

[0101] The data display and export module is used to display the captured TCP data packets and export them as Excel files.

[0102] Various changes and modifications can be made without departing from the spirit and scope of the present invention, and all equivalent technical solutions also fall within the scope of the present invention.

[0103] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referenced to each other.

[0104] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0106] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0108] It should be noted that:

[0109] References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "one embodiment" or "an embodiment" in various places throughout this specification do not necessarily refer to the same embodiment.

[0110] In addition, it should be noted that the shapes and names of the components of the specific embodiments described in this specification may vary. Any equivalent or simple variation based on the structure, features, and principles described in the patent concept of the present invention is included within the scope of protection of the patent. Those skilled in the art of the present invention may make various modifications, additions, or substitutions to the described specific embodiments, and these modifications, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, shall fall within the scope of protection of the present invention.

Claims

1. A data capture and analysis method for charging pile port monitoring, characterized in that: The following steps are involved: Building a packet capture platform, wherein the packet capture platform has a unified user interface; Input the port to be captured through the configuration file, and select the channel through the user interface; The packet capture platform captures the TCP data packet of the port to be captured, and parses the TCP data packet to obtain parsed data; Send the parsed data to the message queue via RabbitMQ and distribute it to the designated router; Display the captured TCP data packets and export them as Excel files.

2. The data capture and analysis method for charging pile port monitoring according to claim 1 is characterized in that: The construction of the packet capture platform includes the following steps: Use Python+Web technology to build a packet capture platform that integrates channel selection, data capture, data display, and data export modules; Use the Tkinter library to build a unified user interface.

3. The data capture and analysis method for charging pile port monitoring according to claim 1 is characterized in that: The step of inputting the port to be captured through the configuration file and selecting the channel through the user interface includes the following steps: Input the ports to be captured through the configuration file, where the ports to be captured include the charging pile port and the target server port; Select a channel name through the pull-down menu on the user interface; Connect to the corresponding RabbitMQ message queue through pika technology according to the channel name.

4. The data capture and analysis method for charging pile port monitoring according to claim 1 is characterized in that: The packet capture platform captures the TCP data packet of the port to be captured, including the following steps: The packet capture platform creates a TCP server through the socket library, and the TCP server monitors the port to be captured and supports multi-threaded processing and concurrent port data capture; The TCP server starts a thread to run the sniff method and captures TCP data packets through the sniff method.

5. The data capture and analysis method for charging pile port monitoring according to claim 1 is characterized in that: The step of parsing the TCP data packet to obtain parsed data comprises the following steps: Use the scapy library to parse TCP packets and extract key information, including source IP, destination IP, source port, destination port, TCP flags, and message content; Check the TCP flag bit by bit to determine the type of TCP data packet; If the TCP data packet contains a message, the message content is recorded in hexadecimal format to obtain parsed data.

6. The data capture and analysis method for charging pile port monitoring according to claim 1, characterized in that: The parsed data is sent to the message queue via RabbitMQ and distributed to the designated router. The following steps are involved: After parsing, the data is interacted with RabbitMQ using the pika library, automatically detecting the RabbitMQ connection status and connection parameters. If the connection is interrupted, reconnection is attempted. The RabbitMQ connection parameters include the host address, port, user name, password, and virtual host. Create a persistent topic switch of topic type; Distribute the parsed data to the specified router in the form of a comma-separated string.

7. The data capture and analysis method for charging pile port monitoring according to claim 6, characterized in that: The packet capture platform provides flexible configuration options, including: the port to be captured, the target IP and target port, the connection parameters of RabbitMQ, and the designated router.

8. The data capture and analysis method for charging pile port monitoring according to claim 1, characterized in that: The packet capture platform has two built-in TCP data storage methods: One is to implement local TCP data storage through logging.handlers.RotatingFileHandler; The other is to send the captured TCP data packets to a remote TCP storage server through the RabbitMQ message queue.

9. A data capture and analysis system for charging pile port monitoring, characterized in that: include: Packet capture platform building module, port confirmation module to be captured, data capture and analysis module, data distribution module, data display and export module, A packet capture platform building module is used to build a packet capture platform with a unified user interface; A port configuration module for capturing, used to input the port to be captured through a configuration file and select a channel through the user interface; A data capture and parsing module, configured to capture the TCP data packets of the port to be captured using the packet capture platform, and parse the TCP data packets to obtain parsed data; The data distribution module is used to send the parsed data to the message queue through RabbitMQ and distribute it to the designated router; The data display and export module is used to display the captured TCP data packets and export them as Excel files.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. A data capture and analysis device for charging pile port monitoring, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images