Data capturing device, method and equipment based on linux system and storage medium

Abstract

According to the data capturing device, method and equipment based on the linux system and the storage medium provided by the invention, kernel log information is recorded in real time through cooperative work of the space monitoring module, the log recording module and the data capturing module, and an information comparison strategy is adopted to ensure that newly generated key logs can be effectively stored, so that the data capturing efficiency is improved. Therefore, the log coverage problem caused by memory limitation is avoided, the state data can be captured when the equipment is in the running state, it is ensured that key information can still be captured in a high-load environment, and the real-time performance and reliability of a system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of system operation and maintenance, and in particular to a data capture device, method, equipment and storage medium based on a Linux system. Background Art

[0002] In a high-traffic network, the amount of packet capture data is huge. Processing and storing this data requires a lot of disk space and efficient data analysis capabilities. Insufficient storage may lead to the loss of key data and affect problem diagnosis. At the same time, real-time packet capture may affect system performance, especially under high load conditions, it may not capture all data packets, thus missing important information.

[0003] In embedded systems, memory and storage resources are limited. When using the dmesg command to view kernel logs, the log buffer may not be able to save all historical information, causing key logs to be overwritten by new logs, affecting problem tracing. When a system that has been running for a long time encounters a problem, it may find that important log information has been overwritten and historical data has been lost. In addition, although the log rolling mechanism can prevent log files from being too large, improper management strategies may lead to the loss of key information.

[0004] In summary, the problems existing in the prior art need to be solved urgently. Summary of the invention

[0005] The present invention provides a data capture device, method, equipment and storage medium based on a Linux system, which are used to solve the defects in the prior art and realize data capture of a collection device.

[0006] The present invention provides a data capture device based on a Linux system, comprising:

[0007] Space monitoring module, logging module and data capture module;

[0008] The space monitoring module is used to monitor the memory space usage rate, and when the memory space usage rate is greater than a preset threshold, a data cleaning operation is performed;

[0009] The logging module is used to record kernel log information;

[0010] The data capture module is used to monitor the working status of the device to be collected, and when the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0011] According to a data capture device based on a Linux system provided by the present invention, the data cleaning operation includes:

[0012] Delete the oldest status data under the storage path;

[0013] as well as

[0014] Delete all status data under the storage path.

[0015] According to a data capture device based on a Linux system provided by the present invention, the log recording module specifically includes: a kernel viewing unit, an information comparison unit and a log generation unit;

[0016] The kernel viewing unit is used to view kernel information of the kernel ring buffer;

[0017] The information comparison unit is used to compare the kernel information with the end part of the kernel information recorded last time to determine new kernel information, and send the new kernel information to the log generation unit;

[0018] The log generating unit is used to generate kernel log information according to the information acquired by the kernel viewing unit.

[0019] According to a data capture device based on a Linux system provided by the present invention, the information comparison unit includes a first database and a second database, the first database is used to store the information obtained by the kernel viewing unit, and the second database is used to store the end part of the information obtained by the kernel viewing unit.

[0020] According to a data capture device based on a Linux system provided by the present invention, the data capture module comprises: a packet capture state judgment unit, a device monitoring unit and a packet capture unit;

[0021] The packet capture state judgment unit is used to check the packet capture state. When the packet capture state is a state in which packet capture can be performed, it is checked whether there is a packet capture program running in the background. If no packet capture program is running, a packet capture instruction is generated.

[0022] The device monitoring unit is used to monitor the working status of the device to be collected in response to the packet capture instruction;

[0023] The packet capture unit is used to capture the status data of the device to be collected within a preset time when the device to be collected is in a running state.

[0024] According to a data capture device based on a Linux system provided by the present invention, the packet capture unit is also used to set the packet capture state to a non-capture state when the device to be collected is in a running state, and to set the packet capture state to a capture state after the preset time.

[0025] According to a data capture device based on a Linux system provided by the present invention, the packet capture unit captures the status data of the device to be collected through a tcpdump packet capture tool.

[0026] The present invention also provides a data capture method based on the Linux system, comprising:

[0027] Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold;

[0028] Record kernel log information through the logging module;

[0029] The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0030] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned data capture methods based on the Linux system is implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data capture method based on the Linux system as described in any one of the above is implemented.

[0032] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned data capture methods based on the Linux system.

[0033] The data capture device, method, equipment and storage medium based on the Linux system provided by the present invention can record kernel log information in real time through the collaborative work of a space monitoring module, a log recording module and a data capture module, and adopt an information comparison strategy to ensure that newly generated key logs can be effectively saved, thereby avoiding log overwriting problems caused by memory limitations, and can capture status data when the device is in operation, ensuring that key information can still be captured in a high-load environment, thereby improving the real-time performance and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a module schematic diagram of the data capture device based on the Linux system provided by the present invention;

[0036] Figure 2 It is a flow chart of the data capture device based on Linux system provided by the present invention;

[0037] Figure 3 It is a flow chart of the data capture method based on Linux system provided by the present invention;

[0038] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In order to solve the problems in the prior art, the present invention proposes a data capture device based on the Linux system to realize data capture of the acquisition device. The following is a description of the data capture device based on the Linux system. Figure 1 As shown, it includes but is not limited to the following modules: space monitoring module 110, log recording module 120 and data capture module 130;

[0041] The space monitoring module 110 is used to monitor the memory space usage rate, and when the memory space usage rate is greater than a preset threshold, a data cleaning operation is performed.

[0042] The space monitoring module 110 adopts a timing check mechanism to periodically evaluate the usage of the memory space. In specific implementation, the module obtains the remaining space information of the specified directory through a system call and compares it with a preset threshold. Once it detects that the memory space usage exceeds the threshold, the module will start a data cleaning operation. The data cleaning operation includes deleting the oldest status data under the storage path, or deleting all status data according to specific needs. This method ensures that the system continues to maintain sufficient storage space under high traffic conditions to avoid data loss due to insufficient storage.

[0043] The logging module 120 is used to record kernel log information.

[0044] The log recording module 120 is composed of a kernel viewing unit, an information comparison unit and a log generation unit. The kernel viewing unit obtains the kernel information of the kernel ring buffer through a system call and passes the information to the information comparison unit. The information comparison unit compares the newly obtained kernel information with the kernel information recorded last time to identify the newly generated log information. All identified new log information will be sent to the log generation unit, which is responsible for formatting the kernel log information and saving it to a predetermined log file. This process effectively avoids duplicate recording of log information while ensuring that key log information will not be lost due to memory limitations.

[0045] The data capture module 130 is used to monitor the working status of the device to be collected, and when the device to be collected is in the running state, the status data of the device to be collected is captured within a preset time.

[0046] The data capture module 130 includes a packet capture state judgment unit, a device monitoring unit and a packet capture unit. The packet capture state judgment unit is responsible for checking the packet capture state. If the current state is that packet capture can be performed, it further verifies whether there is a packet capture program running in the background. If there is no packet capture program in operation, the module will generate a corresponding packet capture instruction. After receiving the packet capture instruction, the device monitoring unit monitors the working state of the device to be collected in real time, and captures the status data of the device through the tcpdump packet capture tool. When the device to be collected is in the running state, the data capture module will capture the status data within the preset time to ensure the effectiveness of real-time data collection. In addition, after the capture is completed, the module sets the packet capture state to a state where packet capture cannot be performed to prevent repeated capture. After the preset time is over, it is restored to a state where packet capture can be performed.

[0047] The data capture device based on the Linux system provided by the present invention records kernel log information in real time through the collaborative work of a space monitoring module, a log recording module and a data capture module, and adopts an information comparison strategy to ensure that newly generated key logs can be effectively saved, thereby avoiding the log overwriting problem caused by memory limitations, and can capture status data when the device is in operation, ensuring that key information can still be captured in a high-load environment, thereby improving the real-time performance and reliability of the system.

[0048] As a further optional embodiment, the data cleaning operation includes:

[0049] Delete the oldest status data under the storage path;

[0050] as well as

[0051] Delete all status data under the storage path.

[0052] In this embodiment, the data cleaning operation can be flexibly adjusted according to the specific needs of the system and the storage resources, mainly including the following two methods:

[0053] Delete the oldest status data under the storage path: After the space monitoring module 110 detects that the storage usage rate exceeds the preset threshold, the system will automatically perform a cleanup operation. In specific implementation, the module will scan the specified storage path, identify and record the oldest status data file. Subsequently, the module deletes the file through a file system call to free up storage space. This method is suitable for scenarios where the system wants to retain recent data for analysis and backtracking to ensure the availability of historical data. Among them, the oldest status data refers to the status data with the earliest generation time.

[0054] Delete all status data under the storage path: Another way to clean up is to directly delete all status data under the storage path. This operation is particularly important when storage resources are extremely tight or the system needs to be fully reset. When implemented, the space monitoring module will call the file system's delete command to clear all data files under the specified directory. This method is suitable for quickly freeing up storage space, but it should be used with caution to avoid losing important historical data.

[0055] Specifically, Figure 2 As shown, thread 1 is responsible for checking the memory space usage rate. It loops with a period of one second, calls the popen pipe operation function, executes the "df-h|grep-w'RootPath$'|awk'{print$5}'" command (RootPath is an absolute path, macro definition setting), and obtains the percentage k of the system space occupied by the current specified path. When the percentage is greater than the parameter value a set (macro setting), the clearing operation will be performed. First, the value of the rollback flag (macro setting) is judged. If it is non-zero, it is true, and only the oldest packet capture data under the storage path FilePath is deleted. The command is "rm$(ls-lt*.pcap|tail-n 1|awk'{print$9}')" until k is not greater than a; if it is zero, it is false, and all packet capture data under the storage path FilePath is directly cleared. The command is "rm*.pacp".

[0056] Through these two flexible cleanup strategies, the system can optimize data management based on real-time storage conditions and business needs to ensure continuous and stable operation.

[0057] As a further optional embodiment, the log recording module specifically includes: a kernel viewing unit, an information comparison unit and a log generation unit;

[0058] The kernel viewing unit is used to view kernel information of the kernel ring buffer;

[0059] The information comparison unit is used to compare the kernel information with the end part of the kernel information recorded last time to determine new kernel information, and send the new kernel information to the log generation unit;

[0060] The log generating unit is used to generate kernel log information according to the information acquired by the kernel viewing unit.

[0061] In this embodiment, the log recording module 120 specifically includes three key units: a kernel viewing unit, an information comparison unit, and a log generation unit. The specific functions and implementation methods of each unit are as follows:

[0062] Kernel View Unit: The kernel view unit is responsible for accessing and viewing kernel information in the kernel ring buffer. When implemented, the module obtains the contents of the current kernel ring buffer through system calls (such as dmesg or reading / proc / kmsg) and stores it in a temporary data structure for subsequent processing. This process can obtain the kernel running status in real time, which is helpful for subsequent troubleshooting and performance monitoring.

[0063] Information comparison unit: The main function of the information comparison unit is to compare the newly acquired kernel information with the end of the last recorded kernel information. By comparing the two data sets, the module can identify new kernel information. In implementation, the module will compare the two sets of data, determine the appearance of new information, and encapsulate the new information and send it to the log generation unit. This comparison mechanism can effectively filter out duplicate information and reduce the redundancy of log records.

[0064] Log generation unit: The log generation unit generates kernel log information based on the new kernel information obtained by the kernel viewing unit. The unit formats the received kernel information into a preset log format (such as JSON or text file) and writes it to the specified log file path. This process ensures the structured storage of kernel log information, which is convenient for subsequent query and analysis.

[0065] Specifically, Figure 2 As shown in the figure, thread 2 is responsible for recording all dmesg kernel information output by the system. Because the information recorded by the system kernel dmesg adopts a rollback mechanism, the oldest content will be overwritten by the new content after the record is full. The thread has two cycles, called cycleA and cycleB, and cycleA contains cycleB. First, open two files. Open LogFile in "a+" mode. This file is used to record all dmesg. Then open LastLineFile in "r+" mode. This file is responsible for recording the last sentence of each scanned dmesg text to cooperate with the program to record all dmesg information.

[0066] Next, it enters cycleA, which is an infinite loop. During this period, it sleeps for 1 second. First, popen is used to execute "dmesg" in pipe file mode, and then it is determined whether there is any content in LastLineFile. If there is, it means that part of the dmesg content has been recorded before. There is no such content in the first cycle when the program is just started. CanWrite will be set to true, and then it will enter cycleB. This loop will continue to read the content output by dmesg sentence by sentence until it is finished. Each sentence read by cycleB will be compared with the content in LastLineFile. At this time, because LastLineFile is empty, this operation is ignored. Then, because canWrite is true, it means that each line of content needs to be recorded, that is, the read content is recorded in LogFile in an appended manner. After the program has recorded all the read lines line by line, the last sentence will be recorded in LastLineFile to mark the starting point of the next record. Here, readers may wonder whether it will be a misjudgment. This can be avoided because there is a timestamp in front of each line of dmesg's output information. Basically, the content of each sentence is different. If the timestamp and content are the same, it will only be dmesg itself that prints repeatedly, which has little effect on the program recording results.

[0067] After cycleB exits and records the last sentence, canWrite is set to false, the thread sleeps for one second, and then cycleA enters the next loop. In this new loop, popen is used to execute "dmesg" in pipe file mode, and then cycleB determines whether there is any content in LastLineFile. The logic of all cycleB loops after the first cycleA is similar, but different from the first time, because after the first cycleA cycle, LastLineFile has content, canWrite is set to false, and then cycleB is entered to read the content sentence by sentence. Each read content will be compared with LastBuffer. If it is inconsistent, it means that the last record of the last cycleA cycle has not been found. If it is consistent, it means that it has been found. The content that continues to be read is the new content generated since the last record. At this time, canWrite is set to true, and then continue skips the cycleB cycle. In the subsequent cycleB loop after the record is found, because canWrite is true, the dmesg content read in each loop will be recorded in LogFile until there is no content to read and cycleB ends. A new record is made again, canWrite is set to false again, and then the thread goes to sleep, waiting for the next cycleA cycle. At this point, the complete record of dmesg information is realized, and the file is LogFile.

[0068] Through the collaborative work of the above three units, the logging module can efficiently and accurately record kernel status changes, provide important reference data for system maintenance, and enhance the stability and maintainability of the system.

[0069] As a further optional embodiment, the information comparison unit includes a first database and a second database, the first database is used to store the information obtained by the kernel viewing unit, and the second database is used to store the end part of the information obtained by the kernel viewing unit.

[0070] In this embodiment, the information comparison unit specifically includes a first database and a second database to achieve efficient information storage and comparison functions. The specific functions and implementation methods of each database are as follows:

[0071] First database: The first database is used to store all kernel information obtained by the kernel viewing unit. This information includes real-time kernel logs, system status information, and other related data. The database can be implemented using a relational database (such as MySQL, SQLite) or a non-relational database (such as MongoDB) to meet the needs of efficient storage and fast retrieval of large amounts of data. After the kernel information is obtained, it is written to the first database in a timely manner to ensure the persistence and traceability of the data.

[0072] Second database: The second database is used to store the end part of the information obtained by the kernel viewing unit, that is, the kernel information recorded last time. The existence of this database enables the information comparison unit to easily access historical records, thereby realizing the comparison of new and old information. Usually, the database can be designed as a lightweight in-memory database (such as Redis) to provide fast access speed and real-time performance.

[0073] Through the collaboration of these two databases, the information comparison unit can quickly extract the latest kernel information from the first database and compare it with the old information in the second database. After the comparison is completed, the second database will be updated to reflect the latest kernel status, thereby ensuring the consistency and accuracy of the data during the information update process. This design not only improves the efficiency of information processing, but also enhances the flexibility and reliability of the system in log management.

[0074] As a further optional embodiment, the data capture module includes: a packet capture state judgment unit, a device monitoring unit and a packet capture unit;

[0075] The packet capture state judgment unit is used to check the packet capture state. When the packet capture state is a state in which packet capture can be performed, it is checked whether there is a packet capture program running in the background. If no packet capture program is running, a packet capture instruction is generated.

[0076] The device monitoring unit is used to monitor the working status of the device to be collected in response to the packet capture instruction;

[0077] The packet capture unit is used to capture the status data of the device to be collected within a preset time when the device to be collected is in a running state.

[0078] In this embodiment, the data capture module includes a packet capture state judgment unit, a device monitoring unit and a packet capture unit, and the specific implementation method is as follows:

[0079] Packet capture status judgment unit: The packet capture status judgment unit is responsible for monitoring the current packet capture status. The unit regularly checks the packet capture status to determine whether it is "capture-ready". If the status is "capture-ready", it further checks whether there is a packet capture program running in the background:

[0080] If no packet capture program is running in the background, the packet capture status judgment unit will generate a corresponding packet capture instruction and pass it to the device monitoring unit to start the data capture process.

[0081] The unit may employ multi-threading technology to ensure real-time monitoring and reduce the impact on system performance.

[0082] Device monitoring unit: The device monitoring unit is responsible for responding to the packet capture instructions from the packet capture status judgment unit. The unit will monitor the working status of the acquisition device in real time to ensure that the device is in normal operation. During the monitoring process, the unit can access the device's operating parameters, such as CPU usage, memory occupancy, etc., to confirm the health status of the device. When the device is in operation, the unit will notify the packet capture unit to start capturing data.

[0083] Packet capture unit: After receiving the notification from the device monitoring unit, the packet capture unit will perform the packet capture operation. The specific steps include:

[0084] Within the preset time, use tcpdump or other packet capture tools to capture the status data of the device to be collected.

[0085] After the packet capture is completed, the packet capture unit will save the captured data and update the relevant status information to ensure that the packet capture status can be restored to the "capture-capable status" in time after the data capture is completed.

[0086] During the packet capture process, the unit can also set a timer to automatically clean up unnecessary intermediate files after the data capture is completed, saving storage space.

[0087] Specifically, refer to Figure 2Thread 3 is responsible for capturing network data and recording some system status values. Here we need to explain some features of the tcpdump packet capture tool. When executing the tcpdump command in the system terminal, add the "-G" option to specify the duration of packet capture. For example, "-Gn" means that the command will last for n seconds after execution and automatically end when the time is up; add the "-w xxx" option to specify the file name "xxx" for storing the captured packet data. There is a feature that as long as the name remains unchanged, the new captured packet data will directly overwrite the old file when the packet capture command is executed again. In other words, if you want to regenerate the captured packet data, you must update the storage file name.

[0088] Thread 3 is generally a loop with an interval of one second. At the beginning of the loop, it first determines whether EnPacp is false. If it is false, it determines whether tcpdump is already executed in the background. If not, it executes "tcpdump udp and host IPv4Addr-G TIMEINTERVAL-W1-iNETWORK-w PcapName&"; IPv4Addr is the IP address of the local machine, obtained by calling GetBrLanInfo(), TIMEINTERVAL is the packet capture duration, macro setting, NETWORK is the network card you want to capture, macro setting, PcapName is more critical, it is essentially a string consisting of the local IP + year, month, day, hour, minute, and second, with the suffix ".pcap", which can be updated to the current time by calling UpdatePcapName(), that is, the file name is updated. As long as EnPacp is always false and PcapName is not updated, there will be only one tcpdump command in the background to execute packet capture, and it will capture packets again after TIMEINTERVAL seconds, but the file will still be the original one. This avoids endless packet capture over time, otherwise the space may be full in a few minutes. Therefore, this can avoid the problem of limited space in some embedded Linux systems; the above GetBrLanInfo() is a self-developed function. By giving the network card name, using the structifreq structure, and calling the ioctl function, you can get the desired local IPV4 address, and then convert it into a string form through inet_ntop(). UpdatePcapName() uses the time() and localtime() functions to obtain the new time value and convert it into a string at the same time, and then combine it with the local IP string to form a new packet transfer data file name.

[0089] The second judgment is to detect whether the acquisition status of the system sound card is in a non-closed state. How to detect it? Here, take a certain brand of sound card as an example. Send a command to the terminal through popen to check the status of the sound card, and then read it. When there is no task, it is generally closed. As long as it is detected to be in a non-closed state, and then isCollectOpened is judged to be false, it means that it is a collection task triggering time, at this time, isCollectOpened is set to true; at this time, it is judged whether EnPacp is false. If it is false, EnPacp is set to true first, so the old PcapName will not be overwritten this time, and it will not move after TIMEINTERVAL ends. Then call UpdatePcapName() to update PcapName and capture again, so that the network data of TIMEINTERVAL seconds before and after the trigger will be obtained because of this task trigger; when the sound card is subsequently detected to be closed, isCollectOpened will be set to false. The process here is to detect the acquisition status trigger, and there is also a detection of the playback status trigger. In other words, as long as the acquisition or playback status becomes active at the beginning, it will become a trigger point, and the program will get the packet capture data before and after this point.

[0090] How can EnPacp be restored to false to restore the original packet capture step that will continuously overwrite files? When the above detection system sound card status becomes active, isCollectOpened is set to true, and a countdown variable EnRecord is opened at the same time, with the same value of TIMEINTERVAL. Because the large loop interval of this thread is 1 second, TIMEINTERVAL will be reduced by one for each loop. If it is zero, EnPacp is set to false, indicating that TIMEINTERVAL seconds have passed since the detection task was triggered, and it is time to resume the normal process.

[0091] During the time when EnRecord decreases from TIMEINTERVAL to 0, some system information will be collected to the DEVICEINFOLOG file every second, such as the sound card acquisition and playback status, volume value, IO status and other status values, to assist in troubleshooting when encountering abnormal tasks.

[0092] As a further optional embodiment, the packet capture unit is also used to set the packet capture state to a non-capture state when the device to be collected is in an operating state, and to set the packet capture state to a capture state after the preset time.

[0093] In this embodiment, in addition to executing data capture, the packet capture unit also has the following functions to manage the packet capture status:

[0094] Packet capture status settings:

[0095] When the device to be collected is confirmed to be in operation, the packet capture unit will immediately set the packet capture status to "unable to capture". At this time, the system will block any new packet capture requests to avoid repeated data capture or resource conflicts.

[0096] The packet capture unit will start the packet capture operation and continuously monitor the status data of the device to be collected within the preset time. During the capture process, any new packet capture request will be rejected to ensure the integrity of the packet capture process.

[0097] State recovery mechanism:

[0098] After the packet capture is completed, the packet capture unit will automatically restore the packet capture status to "capture-ready status" according to the preset time. This process ensures that the system is always ready to accept new packet capture requests.

[0099] Before the status is restored, the packet capture unit can record the currently captured status data and generate corresponding logs for subsequent analysis and troubleshooting.

[0100] Status feedback:

[0101] The packet capture unit can also send information about the current packet capture status to other modules of the system (such as the device monitoring unit) through the status feedback mechanism. This information transmission helps coordinate other operations within the system and ensure the effective use of resources.

[0102] As a further optional embodiment, the packet capturing unit captures the status data of the device to be collected through a tcpdump packet capturing tool.

[0103] The packet capture unit is configured to call the tcpdump tool, which runs in the background and is responsible for capturing the network data packets of the device to be collected in real time. tcpdump is executed in command line form and can effectively handle data capture in a high-traffic environment. Before starting tcpdump, the packet capture unit will configure the network interface, filtering conditions (such as protocol type, target address, etc.) and output file name for packet capture according to the parameters set by the user. In this way, the user can flexibly define the scope and content of the packet capture. When the packet capture status is set to "capture-ready status", the packet capture unit will generate and execute the corresponding tcpdump command to start capturing the status data of the device to be collected. The captured data will be written to the specified output file in real time to ensure the integrity and timeliness of the data.

[0104] The following is a description of the data capture device based on the Linux system provided by the present invention. Figure 3 As shown, the data capture device based on the Linux system described below and the data capture method based on the Linux system described above can refer to each other.

[0105] A data crawling method based on Linux system, comprising:

[0106] Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold;

[0107] Record kernel log information through the logging module;

[0108] The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0109] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the data capture method based on the Linux system, and the method includes:

[0110] Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold;

[0111] Record kernel log information through the logging module;

[0112] The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0113] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0114] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the data capture method based on the Linux system provided by the above methods, the method includes:

[0115] Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold;

[0116] Record kernel log information through the logging module;

[0117] The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0118] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for capturing data based on a Linux system provided by the above methods is implemented, and the method comprises:

[0119] Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold;

[0120] Record kernel log information through the logging module;

[0121] The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

[0122] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data capture device based on Linux system, characterized in that: include: Space monitoring module, logging module and data capture module; The space monitoring module is used to monitor the memory space usage rate, and when the memory space usage rate is greater than a preset threshold, a data cleaning operation is performed; The logging module is used to record kernel log information; The data capture module is used to monitor the working state of the device to be collected, and when the working state of the device to be collected is in a running state, the state data of the device to be collected is captured within a preset time.

2. The data capture device based on Linux system according to claim 1, characterized in that: The data cleaning operation includes: Delete the oldest status data under the storage path; as well as Delete all status data under the storage path; When the memory space usage is greater than a preset threshold, a data cleaning operation is performed.

3. The data capture device based on Linux system according to claim 1, characterized in that: The log recording module specifically includes: a kernel viewing unit, an information comparison unit and a log generation unit; The kernel viewing unit is used to view kernel information of the kernel ring buffer; The information comparison unit is used to compare the kernel information with the end part of the kernel information recorded last time to determine new kernel information, and send the new kernel information to the log generation unit; The log generating unit is used to generate kernel log information according to the information acquired by the kernel viewing unit.

4. The data capture device based on Linux system according to claim 3, characterized in that: The information comparison unit includes a first database and a second database, wherein the first database is used to store the kernel information, and the second database is used to store the end part of the kernel information.

5. The data capture device based on Linux system according to claim 1, characterized in that: The data capture module includes: a packet capture state judgment unit, a device monitoring unit and a packet capture unit; The packet capture state judgment unit is used to check the packet capture state. When the packet capture state is a state in which packet capture can be performed, it is checked whether there is a packet capture program running in the background. If no packet capture program is running, a packet capture instruction is generated. The device monitoring unit is used to monitor the working status of the device to be collected in response to the packet capture instruction; The packet capture unit is used to capture the status data of the device to be collected within a preset time when the device to be collected is in a running state.

6. The data capture device based on Linux system according to claim 5, characterized in that: The packet capture unit is further configured to set the packet capture state to a non-capture state when the device to be collected is in a running state, and to set the packet capture state to a capture state after the preset time.

7. The data capture device based on Linux system according to claim 1, characterized in that: The packet capture unit captures the status data of the device to be collected through the tcpdump packet capture tool.

8. A data capture method based on Linux system, characterized in that: include: Monitor the memory space usage rate through the space monitoring module, and perform a data cleaning operation when the memory space usage rate is greater than a preset threshold; Record kernel log information through the logging module; The working status of the device to be collected is monitored by a data capture module. When the device to be collected is in a running state, the status data of the device to be collected is captured within a preset time.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data capture method based on the Linux system as claimed in claim 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the data capture method based on the Linux system as claimed in claim 8 is implemented.

Images