Data acquisition optimization system and method based on event driving

Abstract

The invention particularly relates to a data acquisition optimization system and method based on event driving. The data acquisition optimization system based on event driving comprises an event monitoring module, a task scheduling module and a data acquisition module. According to the data acquisition optimization method based on event driving, the problems of poor real-time performance, easy data loss and the like during base station data acquisition of a traditional timing task are effectively solved, the real-time performance and integrity of base station data acquisition are remarkably improved, the resource utilization efficiency is improved, and good expandability and flexibility are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition and scheduling, and in particular to an event-driven data acquisition optimization system and method. Background Art

[0002] In today's era of rapidly developing communication networks, base stations, as key nodes in the communication network, are responsible for processing and transmitting massive amounts of data. The accuracy, real-time nature, and completeness of base station data collection are crucial for optimizing and managing communication networks and improving service quality.

[0003] Currently, base station data collection systems generally use a traditional scheduled task approach to schedule data collection. This scheduled task model can ensure data collection to a certain extent when data traffic is relatively stable and regular. However, with the increasing diversity and complexity of communication services, data traffic has become dynamically changing, and this change is significantly uneven.

[0004] Specifically, when the scheduled interval is set to a small amount, such as when executing tasks in hourly intervals, it is common for the corresponding data files to not be generated or transferred at the time of task execution. This results in a local execution by-pass, preventing the acquisition of valid data for this collection task. This not only wastes system resources but can also result in the loss of critical data, hindering subsequent accurate analysis and decision-making regarding base station operating status.

[0005] On the other hand, setting a larger timer interval can, to some extent, avoid the problem of byes due to ungenerated files, but it can also introduce new problems. When tasks are executed, the accumulation of too many data files during the interval can cause local data transmission congestion. This data transmission congestion not only slows down data transmission and affects the real-time nature of data collection, but can also lead to partial data loss when the data volume is large and transmission resources are limited, seriously compromising the integrity of data collection.

[0006] In summary, the traditional scheduled task method faces huge challenges in the controllability of base station data collection and scheduling because it cannot effectively cope with the dynamic and unbalanced changes in data traffic. It is difficult to meet the strict requirements of modern communication networks for the real-time and integrity of base station data collection. A more advanced, flexible and effective scheduling technology is urgently needed to solve these problems.

[0007] Based on the above situation, the present invention proposes an event-driven data acquisition optimization system and method. Summary of the Invention

[0008] In order to overcome the defects of the prior art, the present invention provides a simple and efficient event-driven data acquisition optimization method.

[0009] The present invention is achieved through the following technical solutions:

[0010] An event-driven data acquisition optimization system includes an event monitoring module, a task scheduling module and a data acquisition module;

[0011] Event monitoring module: responsible for creating inotify instances based on the inotify mechanism, registering monitoring items with the kernel to monitor the base station data file storage directory; cyclically reading the inotify event queue and parsing the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module;

[0012] After the event monitoring module creates an inotify instance, it first obtains the inotify file descriptor; then, based on the inotify file descriptor, it uses the system call function to add a monitoring item for the base station data file storage directory to the kernel and sets the monitoring event mask, including the data file creation event;

[0013] Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

[0014] Task scheduling module: After receiving an event notification, it is responsible for generating corresponding collection tasks based on the event type and setting priorities for the tasks; inserting tasks into the task queue according to priority, monitoring the task queue status in real time, and when system resources allow, taking the task with the highest priority from the task queue and assigning it to the data collection module, and recording the task execution status;

[0015] The task scheduling module is responsible for implementing task generation, priority setting, task queue management, task allocation and status recording. The specific implementation methods are as follows:

[0016] (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated;

[0017] (2) Priority setting: Customize the priority of the generated tasks based on the type, size, and custom-defined importance factors of the data files;

[0018] (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue;

[0019] (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

[0020] Data acquisition module: responsible for receiving tasks from the task scheduling module, obtaining the path of the data file to be collected, reading the data file in block reading mode, performing checksum calculation after each data block is read, and comparing it with the checksum information carried in the file:

[0021] If they are consistent, they are sent to the data processing center;

[0022] If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module;

[0023] After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

[0024] The data acquisition module is responsible for implementing data file reading, block transmission, verification calculation and error handling. The specific implementation method is as follows:

[0025] (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size;

[0026] (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission. In the selection of data transmission protocol, the appropriate protocol is selected according to the characteristics of the base station data file and the network environment, such as TCP / IP protocol.

[0027] (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm (such as the CRC-32 algorithm), and the calculated checksum is compared with the checksum information carried by the data file;

[0028] (4) Error handling: If the comparison results are inconsistent, re-read the data block

[0029] The data acquisition module pre-sets a reasonable buffer size, and dynamically adjusts the reasonable buffer size according to the average size of the base station data file and the network bandwidth to balance memory usage and transmission speed.

[0030] An event-driven data acquisition optimization method includes the following steps:

[0031] Step S1: Create an inotify instance based on the inotify mechanism, register a monitoring item with the kernel to monitor the base station data file storage directory; cyclically read the inotify event queue and parse the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module;

[0032] Step S2: After receiving the event notification, the task scheduling module generates a corresponding collection task according to the event type and sets a priority for the task; inserts the task into the task queue according to the priority, monitors the task queue status in real time, and when system resources allow, takes the task with the highest priority from the task queue and assigns it to the data collection module, and records the task execution status;

[0033] Step S3: After receiving the task from the task scheduling module, the data acquisition module obtains the path of the data file to be collected, reads the data file in a block-by-block reading mode, performs a checksum calculation after each data block is read, and compares it with the checksum information carried in the file:

[0034] If they are consistent, they are sent to the data processing center;

[0035] If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module;

[0036] After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

[0037] In step S1, after creating the inotify instance, first obtain the inotify file descriptor; then, based on the inotify file descriptor, use the system call function to add a monitoring item for the base station data file storage directory to the kernel, and set the monitoring event mask, including the data file creation event;

[0038] Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

[0039] In step S2, the specific implementation is as follows:

[0040] (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated;

[0041] (2) Priority setting: Customize the priority of the generated tasks based on the type, size, and custom-defined importance factors of the data files;

[0042] (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue;

[0043] (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

[0044] In step S3, the specific implementation is as follows:

[0045] (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size;

[0046] (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission. In the selection of data transmission protocol, the appropriate protocol is selected according to the characteristics of the base station data file and the network environment, such as TCP / IP protocol.

[0047] (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm (such as the CRC-32 algorithm), and the calculated checksum is compared with the checksum information carried by the data file;

[0048] (4) Error handling: If the comparison results are inconsistent, re-read the data block

[0049] In step S3, a reasonable buffer size is pre-set, and the reasonable buffer size is dynamically adjusted according to the average size of the base station data file and the network bandwidth to balance the memory usage and the transmission speed.

[0050] The beneficial effects of the present invention are: this event-driven data acquisition optimization method effectively solves the problems of poor real-time performance and easy data loss in traditional scheduled tasks when collecting base station data, significantly improves the real-time performance and integrity of base station data collection, improves resource utilization efficiency, and has good scalability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 This is a schematic diagram of the event-driven workflow of the present invention. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0054] The event-driven data acquisition optimization system includes an event monitoring module, a task scheduling module and a data acquisition module;

[0055] Event monitoring module: responsible for creating inotify instances based on the inotify mechanism, registering monitoring items with the kernel to monitor the base station data file storage directory; cyclically reading the inotify event queue and parsing the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module;

[0056] After the event monitoring module creates an inotify instance, it first obtains the inotify file descriptor; then, based on the inotify file descriptor, it uses the system call function to add a monitoring item for the base station data file storage directory to the kernel and sets the monitoring event mask, including the data file creation event;

[0057] Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

[0058] Task scheduling module: After receiving an event notification, it is responsible for generating corresponding collection tasks based on the event type and setting priorities for the tasks; inserting tasks into the task queue according to priority, monitoring the task queue status in real time, and when system resources allow, taking the task with the highest priority from the task queue and assigning it to the data collection module, and recording the task execution status;

[0059] The task scheduling module is responsible for implementing task generation, priority setting, task queue management, task allocation and status recording. The specific implementation methods are as follows:

[0060] (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated;

[0061] (2) Priority setting: Customize the priority of the generated task based on the type, size, and custom-defined importance of the data file; for example, set a higher priority for urgent data files and a lower priority for common data files;

[0062] (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue;

[0063] (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

[0064] Data acquisition module: responsible for receiving tasks from the task scheduling module, obtaining the path of the data file to be collected, reading the data file in block reading mode, performing checksum calculation after each data block is read, and comparing it with the checksum information carried in the file:

[0065] If they are consistent, they are sent to the data processing center;

[0066] If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module;

[0067] After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

[0068] The data acquisition module is responsible for implementing data file reading, block transmission, verification calculation and error handling. The specific implementation method is as follows:

[0069] (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size;

[0070] (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission. In the selection of data transmission protocol, the appropriate protocol is selected according to the characteristics of the base station data file and the network environment, such as TCP / IP protocol.

[0071] (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm (such as the CRC-32 algorithm), and the calculated checksum is compared with the checksum information carried by the data file;

[0072] (4) Error handling: If the comparison results are inconsistent, re-read the data block

[0073] The data acquisition module pre-sets a reasonable buffer size, and dynamically adjusts the reasonable buffer size according to the average size of the base station data file and the network bandwidth to balance memory usage and transmission speed.

[0074] The event-driven data acquisition optimization method includes the following steps:

[0075] Step S1: Create an inotify instance based on the inotify mechanism, register a monitoring item with the kernel to monitor the base station data file storage directory; cyclically read the inotify event queue and parse the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module;

[0076] Step S2: After receiving the event notification, the task scheduling module generates a corresponding collection task according to the event type and sets a priority for the task; inserts the task into the task queue according to the priority, monitors the task queue status in real time, and when system resources allow, takes the task with the highest priority from the task queue and assigns it to the data collection module, and records the task execution status;

[0077] Step S3: After receiving the task from the task scheduling module, the data acquisition module obtains the path of the data file to be collected, reads the data file in a block-by-block reading mode, performs a checksum calculation after each data block is read, and compares it with the checksum information carried in the file:

[0078] If they are consistent, they are sent to the data processing center;

[0079] If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module;

[0080] After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

[0081] In step S1, after creating the inotify instance, first obtain the inotify file descriptor; then, based on the inotify file descriptor, use the system call function to add a monitoring item for the base station data file storage directory to the kernel, and set the monitoring event mask, including the data file creation event;

[0082] Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

[0083] In step S2, the specific implementation is as follows:

[0084] (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated;

[0085] (2) Priority setting: Customize the priority of the generated tasks based on the type, size, and custom-defined importance factors of the data files;

[0086] (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue;

[0087] (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

[0088] In step S3, the specific implementation is as follows:

[0089] (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size;

[0090] (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission. In the selection of data transmission protocol, the appropriate protocol is selected according to the characteristics of the base station data file and the network environment, such as TCP / IP protocol.

[0091] (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm (such as the CRC-32 algorithm), and the calculated checksum is compared with the checksum information carried by the data file;

[0092] (4) Error handling: If the comparison results are inconsistent, re-read the data block

[0093] In step S3, a reasonable buffer size is pre-set, and the reasonable buffer size is dynamically adjusted according to the average size of the base station data file and the network bandwidth to balance the memory usage and the transmission speed.

[0094] Compared with existing technologies, this event-driven data acquisition optimization method has the following characteristics:

[0095] (1) Improved real-time performance: By replacing traditional scheduled tasks with event-driven systems, the collection task can be started the moment the base station data file is generated, significantly shortening the time interval between data generation and collection, and providing timely data support for base station monitoring scenarios with high real-time requirements.

[0096] (2) Guaranteed integrity: Accurately perceive data file generation events, reasonably arrange collection tasks, and avoid excessive data accumulation; the block reading and checksum method strictly guarantees data accuracy from triggering collection to transmission, effectively avoiding data loss.

[0097] (3) Efficient resource utilization: triggering data collection based on data generation events avoids invalid collection attempts and reduces resource waste; reasonable task scheduling and data transmission strategies, such as dynamically adjusting the buffer size, enable more reasonable allocation and utilization of network resources and system memory, reducing system operating costs.

[0098] (4) Enhanced scalability and flexibility: The inotify mechanism makes it easy to expand monitoring directories and file types, and the event-driven architecture facilitates the integration of other event processing logic. The system can easily adapt to changes in base station data collection requirements without the need for large-scale reconstruction.

[0099] The embodiment described above is only one specific implementation of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. An event-driven data acquisition and optimization system, characterized by: Including event monitoring module, task scheduling module and data acquisition module; Event monitoring module: responsible for creating inotify instances based on the inotify mechanism, registering monitoring items with the kernel to monitor the base station data file storage directory; cyclically reading the inotify event queue and parsing the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module; Task scheduling module: responsible for generating corresponding collection tasks according to event types after receiving event notifications, and setting priorities for tasks; Insert tasks into the task queue according to their priority, monitor the task queue status in real time, and when system resources allow, take out the task with the highest priority from the task queue and assign it to the data acquisition module, and record the task execution status; Data acquisition module: responsible for receiving tasks from the task scheduling module, obtaining the path of the data file to be collected, reading the data file in block reading mode, performing checksum calculation after each data block is read, and comparing it with the checksum information carried in the file: If they are consistent, they are sent to the data processing center; If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module; After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

2. The event-driven data acquisition and optimization system according to claim 1, characterized in that: After the event monitoring module creates an inotify instance, it first obtains the inotify file descriptor; then, based on the inotify file descriptor, it uses the system call function to add a monitoring item for the base station data file storage directory to the kernel and sets the monitoring event mask, including the data file creation event; Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

3. The event-driven data acquisition and optimization system according to claim 1, characterized in that: The task scheduling module is responsible for implementing task generation, priority setting, task queue management, task allocation and status recording. The specific implementation methods are as follows: (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated; (2) Priority setting: Customize the priority of the generated tasks based on the type, size, and custom-defined importance factors of the data files; (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue; (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

4. The event-driven data acquisition and optimization system according to claim 1, characterized in that: The data acquisition module is responsible for implementing data file reading, block transmission, verification calculation and error handling. The specific implementation method is as follows: (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size; (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission; (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm, and the calculated checksum is compared with the checksum information carried by the data file; (4) Error handling: If the comparison results are inconsistent, reread the data block.

5. The event-driven data acquisition and optimization system according to claim 4, characterized in that: The data acquisition module pre-sets a reasonable buffer size, and dynamically adjusts the reasonable buffer size according to the average size of the base station data file and the network bandwidth to balance memory usage and transmission speed.

6. An event-driven data acquisition optimization method, characterized by: The following steps are involved: Step S1: Create an inotify instance based on the inotify mechanism, register a monitoring item with the kernel to monitor the base station data file storage directory; cyclically read the inotify event queue and parse the captured events. If a data file generation event is detected, the event notification is sent to the task scheduling module; Step S2: After receiving the event notification, the task scheduling module generates a corresponding collection task according to the event type and sets a priority for the task; inserts the task into the task queue according to the priority, monitors the task queue status in real time, and when system resources allow, takes the task with the highest priority from the task queue and assigns it to the data collection module, and records the task execution status; Step S3: After receiving the task from the task scheduling module, the data acquisition module obtains the path of the data file to be collected, reads the data file in a block-by-block reading mode, performs a checksum calculation after each data block is read, and compares it with the checksum information carried in the file: If they are consistent, they are sent to the data processing center; If there is any inconsistency, re-read and retry up to 3 times. If it still fails after 3 retries, record the error information and report it to the task scheduling module; After completing the reading of all data blocks of a data file and sending them successfully, a task completion notification is sent to the task scheduling module.

7. The event-driven data acquisition optimization method according to claim 6, characterized in that: In step S1, after creating the inotify instance, first obtain the inotify file descriptor; then, based on the inotify file descriptor, use the system call function to add a monitoring item for the base station data file storage directory to the kernel, and set the monitoring event mask, including the data file creation event; Finally, a loop structure is used to read event data from the inotify event queue through a system call function, and the read event data is parsed to identify the specific event type.

8. The event-driven data acquisition optimization method according to claim 6, characterized in that: In step S2, the specific implementation is as follows: (1) Task generation: Based on the event notification sent by the event monitoring module, the event type is determined. If it is a data file generation event, the corresponding base station data collection task is generated; (2) Priority setting: Customize the priority of the generated tasks based on the type, size, and custom-defined importance factors of the data files; (3) Task queue management: Tasks with set priorities are inserted into the task queue in order of priority, and are stored and managed using a priority queue data structure to ensure that high-priority tasks are always at the head of the queue; (4) Task allocation and status recording: Real-time monitoring of the task queue status. When the system's CPU usage, memory free space, and network bandwidth utilization meet the task execution conditions, the highest priority task is taken from the head of the task queue and assigned to the data acquisition module. The execution status information of the task is recorded, including the allocation time and task ID. During the task execution process, the task status is updated in real time, including being executed, completed, and failed.

9. The event-driven data acquisition optimization method according to claim 6, characterized in that: In step S3, the specific implementation is as follows: (1) Data file reading: According to the path of the data file to be collected obtained from the task scheduling module, the file system operation function is used to open the data file, and the data file content is read block by block in the pre-set block size; (2) Block transmission: The read data blocks are sent to the data processing center through the selected data transmission protocol. During the transmission process, the transmission rate is dynamically adjusted according to the network congestion to ensure the stability and efficiency of data transmission; (3) Checksum calculation: For each read data block, a checksum is calculated using a preset checksum algorithm, and the calculated checksum is compared with the checksum information carried by the data file; (4) Error handling: If the comparison results are inconsistent, reread the data block.

10. The event-driven data acquisition optimization method according to claim 6, characterized in that: In step S3, a reasonable buffer size is pre-set, and the reasonable buffer size is dynamically adjusted according to the average size of the base station data file and the network bandwidth to balance the memory usage and the transmission speed.

Images