Real-time data acquisition task scheduling method and system

Abstract

The invention discloses a real-time data acquisition task scheduling method and system, and relates to the technical field of scheduling automatic data acquisition. The method comprises the steps that a first data acquisition framework is constructed, and the first data acquisition framework comprises a first management unit and a plurality of first data acquisition units; the first management unit is used for first task scheduling; the first data acquisition unit performs first data acquisition in response to the first task scheduling. According to the method, container technical characteristics are utilized, a containerized data acquisition architecture in which container management nodes and acquisition containers are coordinated is constructed, the problem of large-scale access of multi-element new industrial form main bodies is solved, the flexibility and reliability of data acquisition and processing are improved, and the basic supporting capacity of a regulation and control system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of scheduling automated data collection, and in particular to a real-time data collection task scheduling method and system. Background Art

[0002] As the power load continues to grow, new factors such as adjustable loads, electric vehicles, distributed photovoltaics, new energy storage, microgrids, and resource aggregators have entered the stage of large-scale development. Data types and access methods are showing a trend of heterogeneity and complexity, and the scale of data is exponentially growing. The existing control system has problems such as insufficient flexibility in data collection architecture, independent deployment mode of multi-subject collection leading to a step-by-step increase in hardware resources and operation and maintenance costs, and the inability to dynamically expand collection performance online. There are still certain limitations in the ability to flexibly access multi-heterogeneous new business entities. Summary of the invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides a real-time data acquisition task scheduling method and system to solve the problems that the existing data acquisition architecture cannot be elastically expanded online and lacks effective security isolation between different acquisition applications.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a real-time data acquisition task scheduling method, comprising: Constructing a first data acquisition framework, wherein the first data acquisition framework includes a first management unit and a plurality of first data acquisition units; The first management unit is used for first task scheduling; The first data collection unit performs first data collection in response to the first task scheduling.

[0006] As a preferred solution of the real-time data acquisition task scheduling method of the present invention, the first management unit includes: The first task scheduling is performed by controlling the first statistical module, the first monitoring module and the first control module through the first start-stop management mechanism.

[0007] As a preferred solution of the real-time data acquisition task scheduling method of the present invention, the first start-stop management mechanism includes: Generate a first control instruction according to the data acquired by the first statistical module and the first monitoring module; The first control module performs start and stop management according to the first control instruction; The first control instructions include control instructions for several working conditions.

[0008] As a preferred solution of the real-time data acquisition task scheduling method of the present invention, wherein: the several working conditions at least include a first working condition, a second working condition and a third working condition; When any of the first data acquisition units fails, it is recorded as the first working condition; When the number of the running first data acquisition units is greater than the required number of the first data acquisition units, it is recorded as the second working condition; When the number of the running first data acquisition units is not greater than the required number of the first data acquisition units, it is recorded as the third working condition.

[0009] As a preferred solution of the real-time data acquisition task scheduling method of the present invention, wherein: the first data acquisition unit includes: generating a first data acquisition logic in response to the first task scheduling; The first data collection logic is a logic designed based on the first communication module, the first parsing module and the first message processing module; The first data collection is performed by implementing the first data collection logic through the first communication module, the first parsing module and the first message processing module.

[0010] As a preferred solution of the real-time data acquisition task scheduling method of the present invention, the first control instruction further includes: When in the first working condition, the control instruction for the first working condition includes a first screening instruction and a first allocation instruction; In the first working condition, the first data acquisition unit that fails sends a first screening instruction to the first management unit, and the first management unit performs a first screening and a second screening in response to the first screening instruction, and generates a first allocation instruction; The remaining several first data acquisition units except the faulty first data acquisition unit perform a reallocation operation according to the first allocation instruction.

[0011] As a preferred solution of the real-time data acquisition task scheduling method described in the present invention, wherein: the first data acquisition unit also includes: any first data acquisition unit is only responsible for the acquisition of one type of business data.

[0012] In a second aspect, the present invention provides a real-time data acquisition task scheduling system, comprising: A framework construction module, used to construct a first data acquisition framework, wherein the first data acquisition framework includes a first management unit and a plurality of first data acquisition units; The first management unit is used for first task scheduling; The first data collection unit performs first data collection in response to the first task scheduling.

[0013] In a third aspect, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the real-time data acquisition task scheduling method are implemented.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the real-time data acquisition task scheduling method.

[0015] Compared with the prior art, the present invention has the following beneficial effects: the present invention constructs a first data acquisition framework, and performs a first task scheduling according to a first management unit of the first data acquisition framework; the first data acquisition unit of the first data acquisition framework performs the first data acquisition in response to the first task scheduling. When counting the number of collection tasks, the corresponding number of collection containers is elastically expanded to increase the data access volume; and the collection tasks are evenly distributed to the corresponding collection containers to solve the problem of dynamic migration of collection tasks when the container fails; the flexibility and reliability of data collection and the basic support capabilities of the control system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 A method flow chart of a real-time data acquisition task scheduling method and system according to an embodiment of the present invention; Figure 2 A data acquisition architecture diagram of a real-time data acquisition task scheduling method and system according to an embodiment of the present invention; Figure 3 A schematic diagram of elastic expansion of a collection container of a real-time data collection task scheduling method and system according to an embodiment of the present invention; Figure 4 The present invention is a schematic diagram of the internal structure of a collection container of a real-time data collection task scheduling method and system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0020] Example 1 Reference Figure 1~Figure 4 , is an embodiment of the present invention, which provides a real-time data acquisition task scheduling method.

[0021] It should be noted that as data types and access methods show a trend of heterogeneity and complexity, the scale of data is growing exponentially. The existing control system has problems such as insufficient flexibility in data collection architecture, a step-by-step increase in hardware resources and operation and maintenance costs due to the independent deployment mode of multiple subject collection, and the inability to dynamically expand collection performance online. It has certain limitations in the ability to flexibly access multiple heterogeneous new business entities.

[0022] The present application provides a method that can effectively solve the above-mentioned problems. Next, we will describe in detail how to implement the real-time data acquisition task scheduling method in combination with multiple embodiments. like Figure 1 As shown, the real-time data collection task scheduling method includes: S100: constructing a first data acquisition framework, the first data acquisition framework including a first management unit and a plurality of first data acquisition units; S200: The first management unit is used for first task scheduling; S300: The first data acquisition unit performs first data acquisition in response to the first task scheduling.

[0023] In an optional embodiment, the first data collection framework can be constructed using blockchain technology, and the transparency and immutability of the data collection task can be ensured through distributed ledger technology to improve the credibility of the data collection task.

[0024] In another optional embodiment, the first data collection framework may also use virtual machine technology to build an efficient and flexible data collection framework to meet the flexible access needs of diverse and heterogeneous new business entities.

[0025] In a preferred embodiment, Figure 2 As shown, the first data collection framework can be constructed using container technology. Based on the rapid deployment, virtualization, resource isolation and other characteristics of container technology, a data collection and control system that can meet the flexible access needs of diverse and heterogeneous new business entities is constructed. The system includes a management node corresponding to the first management unit and a collection container data collection cluster composed of several first data collection units working in conjunction with the management node.

[0026] It should be noted that the coordination between the management node and the collection container data collection cluster can support the online dynamic expansion of the collection capacity and collection type of the data collection and control system. The management node implements the scheduling of pre-collection tasks and the on-demand elastic expansion of the collection container. A lightweight stateless collection module is deployed in the collection container to realize the separate collection and independent operation of the corresponding collection objects.

[0027] In the embodiment of the present application, the first management unit includes: The first task scheduling is performed by controlling the first statistical module, the first monitoring module and the first control module through the first start-stop management mechanism.

[0028] In an optional embodiment, if Figure 3 As shown, the first statistical module can be used for collection task statistics, the first monitoring module can be used for horizontal expansion monitoring, and the first control module can be used for elastic scaling control to control the collection container to collect collection object data, such as photovoltaic data and other new business data.

[0029] In an optional embodiment, the collection objects may include substations, switch stations, distribution rooms, smart substations, photovoltaics, wind power, energy storage, charging piles, adjustable loads and other new business entities. The first start-stop management mechanism collects business data of new business entities by counting collection tasks, monitoring the indicators of collection tasks, and notifying elastic scaling control to dynamically adjust the number of collection containers.

[0030] In the embodiment of the present application, the first start-stop management mechanism includes: Generate a first control instruction according to the data acquired by the first statistical module and the first monitoring module; The first control module performs start and stop management according to the first control instruction; The first control instruction includes control instructions for several working conditions.

[0031] In an optional embodiment, the management node monitors the task changes of different collection objects by counting collection tasks, obtains data information of different collection objects, and provides collection task indicator information for horizontal expansion monitoring; the data information may be information such as the number of plants and stations, the number of channels, the number of measurement points, etc. of the collection objects.

[0032] In an optional embodiment, the indicators of the collection tasks are continuously monitored, the required number of containers is calculated, and the elastic scaling control is notified to dynamically adjust the number of collection containers. The elastic scaling control is the specific executor of the dynamic expansion of the collection containers. It obtains the required number of containers based on horizontal expansion monitoring and compares it with the currently running number of containers. The elastic expansion of the collection containers is controlled based on the comparison results.

[0033] In the embodiment of the present application, the plurality of operating conditions include at least a first operating condition, a second operating condition and a third operating condition; When any of the first data acquisition units fails, it is recorded as the first working condition; When the number of the running first data acquisition units is greater than the required number of the first data acquisition units, it is recorded as the second working condition; When the number of the running first data acquisition units is not greater than the required number of the first data acquisition units, it is recorded as the third working condition.

[0034] Exemplarily, when the data acquisition control system distributes the acquisition tasks evenly to each acquisition container through dynamic allocation management, if any container fails, the tasks managed by it will be reallocated to other containers that are working normally, which is regarded as the first operating condition.

[0035] Exemplarily, the elastic scaling control dynamically adjusts the number of collection containers according to the control instruction. If the number of containers required is more than the number of currently running containers, a new container is started, which is regarded as the second working condition.

[0036] Exemplarily, if the number of containers required is not more than the number of currently running containers, the redundant containers are stopped, which is regarded as the third working condition.

[0037] In an optional embodiment, the plurality of operating conditions further include a fourth operating condition; The fourth working condition includes but is not limited to the following situations: when the system detects changes in the software operating environment, the system automatically adjusts the data collection strategy to adapt to the changes in the operating environment to ensure the accuracy and stability of data collection.

[0038] In an optional embodiment, when the system detects that the operating status of key equipment is abnormal, such as the host machine crashes or key components of the operating system are damaged, the system will automatically switch to the fourth operating state. In this operating state, the system will quickly diagnose and isolate key components through pre-set emergency procedures, and send alarm information to maintenance personnel to ensure that the stable operation of the system is not affected.

[0039] In another optional embodiment, the fourth operating condition also includes dynamic adjustment of the data acquisition frequency to adapt to needs of different urgency levels, thereby minimizing the impact on normal operation while ensuring data integrity.

[0040] In the embodiment of the present application, the first data acquisition unit includes: generating a first data collection logic in response to a first task schedule; The first data collection logic is a logic designed based on the first communication module, the first parsing module and the first message processing module; The first data collection is performed by implementing the first data collection logic through the first communication module, the first parsing module and the first message processing module.

[0041] In an optional embodiment, if Figure 4 As shown, the first data collection unit serves as a collection container, and is used for data collection and processing in response to the task scheduling of the management node. The first communication module is used for link communication, the first parsing module is used for protocol parsing, and the first message processing module is used for message processing.

[0042] It should be noted that a lightweight, stateless collection module is deployed in the collection container. There are no basic services such as real-time libraries and relationship libraries in the container. Business programs access external resources through network client interfaces.

[0043] It should also be noted that Figure 4 The SCADA data processing shown in the figure refers to the process of receiving, processing and storing data from the collection container in the Supervisory Control and Data Acquisition (SCADA) system, which can then be used for further analysis or as input for control logic. For example, in the power system, SCADA can help monitor the status of the power grid and automatically adjust the power generation or distribution strategy as needed.

[0044] In an optional embodiment, the collection container integrates link communication, protocol parsing module and message processing into a collection application to implement specific data collection functions.

[0045] In an optional embodiment, the first communication module is used for link communication, realizes network communication with the station device in the container, and is responsible for sending and receiving TCP messages. The management node allocates a unique virtual IP address to a group of containers through the container network service; by collecting the container network service of the container link communication, a group of containers are provided as a whole to the outside world; the station device determines the access container group by accessing the virtual IP address of the container network service.

[0046] In an optional embodiment, when the link communication is a client of TCP communication, the IP address of the peer communication is the address of the host machine where the collection container is located.

[0047] In another optional embodiment, when the link communication is external communication of the TCP server, based on the container network service provided by the container platform, a group of containers is used as a container network service as a whole to provide service functions to the outside world. The container network service allows a unique virtual IP address to be assigned to a group of containers. The station-side device can directly access the virtual IP address of the container network service to access the container group without having to care about the IP address of the host machine where each collection container is located. The container network service forwards the accessed traffic to the corresponding container in the container group, similar to the load balancing device at the front end of the container.

[0048] In an optional embodiment, the first parsing module is used for protocol parsing to parse and assemble data collection task messages assigned to the collection container. The protocol parsing is assembled into a protocol library by multiple protocol plug-ins, supporting IEC101, IEC104, edge gateway plug-and-play protocols, etc. Each plug-in can run independently without interfering with each other. Each type of collection container runs the corresponding protocol plug-in according to the different types of station-end devices.

[0049] In an optional embodiment, the collection container sends and receives the collected data through a message bus, and ensures that each data is mapped to an independent message channel, thereby realizing on-demand diversion of real-time data for subscription by different data processing containers.

[0050] It should be noted that the message subject of the message bus for sending and receiving collected data in the embodiment of the present application can be composed of information such as container type and container serial number.

[0051] In the embodiment of the present application, the first control instruction also includes: When in the first working condition, the control instruction for the first working condition includes a first screening instruction and a first allocation instruction; When in the first working condition, the first data acquisition unit having a fault sends a first screening instruction to the first management unit, and the first management unit performs a first screening and a second screening in response to the first screening instruction, and generates a first allocation instruction; The remaining several first data acquisition units except the faulty first data acquisition unit perform a reallocation operation according to the first allocation instruction.

[0052] In an optional embodiment, the basic unit of the collection task is the front channel, and the collection task dynamic allocation strategy includes the following steps: Count the number of channels of the corresponding front-end application in the system; Get the number of currently running collection containers and evenly distribute the channels to the corresponding collection containers according to the load balancing algorithm; When a container fails, the channels it manages will be reallocated to other normal containers.

[0053] Furthermore, the steps of channel allocation are as follows: Step 1: read the front channel model and filter out the channels whose containers are empty; Step 2: Obtain the currently running container information and select the collection container that can receive the newly added channel; Step three: Use a round-robin load balancing algorithm to allocate the front channels to the corresponding collection containers in turn, so as to keep the number of channels managed by each container relatively balanced.

[0054] Step 4: To avoid collection service anomalies caused by unnecessary channel switching between containers, channels are generally kept fixed after being assigned to containers. When a collection container fails, the management node immediately pulls up a new container and takes over all channels of the failed container.

[0055] For example, if the access capacity of a single collection container is to access the real-time data of 100 distributed power sources, when there are 300 distributed power sources in the system, the management node automatically pulls up 3 collection containers according to the current task volume, and distributes these collection tasks evenly to the 3 containers according to the load balancing strategy; when the collection tasks increase to 400, the management node automatically pulls up the fourth collection container and distributes the newly added 100 collection tasks to the fourth container for processing.

[0056] In the embodiment of the present application, the first data collection unit also includes: any first data collection unit is only responsible for collecting one type of business data.

[0057] In an optional embodiment, by constructing a data collection architecture in which container management nodes and collection containers cooperate with each other, the online dynamic expansion of the data collection capacity and collection types of the control system is supported; the management node elastically expands the corresponding number of collection containers according to the number of collection tasks counted, thereby increasing the data access volume; the management node evenly distributes the collection tasks to the corresponding collection containers through a load balancing algorithm, thereby solving the problem of dynamic migration of collection tasks when the container fails; when the link communication in the collection container is used as a TCP server to communicate with the outside, based on the container network service provided by the container platform, a group of containers are used as a container network service as a whole to provide services to the outside, and a group of containers provide a unique virtual IP address to the outside, thereby solving the problem that the IP address of the external service cannot be determined when the container switches between different host machines; the characteristics of container rapid deployment, virtualization, resource isolation, etc. are utilized to solve the problem of large-scale access of multiple new business entities, improve the flexibility and reliability of data collection, and enhance the basic support capabilities of the control system.

[0058] Example 2 Reference Figure 1~Figure 4 , is an embodiment of the present invention. This embodiment is different from the first embodiment in that it provides a real-time data acquisition task scheduling system.

[0059] It should be noted that the technical solution of the real-time data acquisition task scheduling system and the technical solution of the above-mentioned real-time data acquisition task scheduling method belong to the same concept. For the details not described in detail in the technical solution of the real-time data acquisition task scheduling system in this embodiment, please refer to the description of the technical solution of the above-mentioned real-time data acquisition task scheduling method.

[0060] In this embodiment, a real-time data acquisition task scheduling system includes: A framework construction module, used to construct a first data collection framework, the first data collection framework includes a first management unit and a plurality of first data collection units; The first management unit is used for first task scheduling; The first data collection unit performs first data collection in response to the first task scheduling.

[0061] In one embodiment, the real-time data acquisition task scheduling system may perform the following steps: The first task scheduling is performed by controlling the first statistical module, the first monitoring module and the first control module through the first start-stop management mechanism.

[0062] Generate a first control instruction according to the data acquired by the first statistical module and the first monitoring module; The first control module performs start and stop management according to the first control instruction; The first control instruction includes control instructions for several working conditions.

[0063] The plurality of operating conditions include at least a first operating condition, a second operating condition and a third operating condition; When any of the first data acquisition units fails, it is recorded as the first working condition; When the number of the running first data acquisition units is greater than the required number of the first data acquisition units, it is recorded as the second working condition; When the number of the running first data acquisition units is not greater than the required number of the first data acquisition units, it is recorded as the third working condition.

[0064] The first data acquisition unit generates a first data acquisition logic in response to the first task scheduling; The first data collection logic is a logic designed based on the first communication module, the first parsing module and the first message processing module; The first data collection is performed by implementing the first data collection logic through the first communication module, the first parsing module and the first message processing module.

[0065] Any first data collection unit is only responsible for collecting one type of business data.

[0066] Further, when in the first working condition, the control instruction for the first working condition includes a first screening instruction and a first allocation instruction; When in the first working condition, the fault first data acquisition unit sends a first screening instruction to the first management unit, and the first management unit performs a first screening and a second screening in response to the first screening instruction, and generates a first allocation instruction; The remaining several first data acquisition units except the faulty first data acquisition unit perform a reallocation operation according to the first allocation instruction.

[0067] This embodiment further provides an electronic device, which is applicable to a method for scheduling a real-time data acquisition task, and includes: A memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement a real-time data acquisition task scheduling method as proposed in the above embodiment.

[0068] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, a real-time data acquisition task scheduling method as proposed in the above embodiment is implemented.

[0069] The storage medium proposed in this embodiment and the method for implementing a real-time data acquisition task scheduling proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0070] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer's floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccess Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0071] Example 3 In the exemplary embodiment, this embodiment takes the acquisition of distributed power data by a control system as an example, and provides an online elastically scalable distributed power data acquisition solution to verify the feasibility of the present invention.

[0072] In this embodiment, a control system is constructed to collect distributed power data, including a distributed power data collection management node and several distributed power data collection containers; the distributed power data collection management node is used for online elastic scaling of the data collection container, and dynamic allocation of data collection tasks among several collection containers; the distributed power data collection container is used to receive the collection tasks assigned by the management node, and complete the real-time access of specific distributed power data.

[0073] In this embodiment, the distributed data collection container can access the real-time data of 100 distributed power sources.

[0074] In an exemplary embodiment, for example, the current collection task index is 400, and the number of currently running containers is 4. By counting the data information such as the number of plants, the number of channels, the number of measuring points, etc. of the distributed power collection objects, the collection task index needs to be increased from 400 to 402 based on the data information; at this time, the number of containers that need to be started is 5; and the number of currently running containers is 4. According to the comparison result, 1 container is elastically expanded, and the total number of running containers is expanded to 5 to complete distributed data collection.

[0075] In another exemplary embodiment, for example, the current collection task index is 397, and the number of currently running containers is 4. By counting the data information such as the number of plants, the number of channels, the number of measuring points of the distributed power supply collection objects, the collection task index needs to be reduced from 397 to 296 based on the data information; at this time, the number of containers that need to be started is 3; and the number of currently running containers is 4. According to the comparison result, 1 container is flexibly reduced, and the total number of running containers is reduced to 3 to complete distributed data collection.

[0076] This embodiment dynamically deploys collection containers through the coordinated cooperation between the distributed power data collection management node and the collection container, and can expand the collection capacity of the control system for distributed power data online. Under the original system architecture, it takes several hours to add front-end servers for capacity expansion. The present invention can make the container expansion efficiency within minutes, which significantly improves the efficiency of data collection and capacity expansion of the control system.

[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A real-time data acquisition task scheduling method, characterized in that: include: Constructing a first data acquisition framework, wherein the first data acquisition framework includes a first management unit and a plurality of first data acquisition units; The first management unit is used for first task scheduling; The first data collection unit performs first data collection in response to the first task scheduling.

2. The real-time data acquisition task scheduling method according to claim 1, characterized in that: The first management unit includes: The first task scheduling is performed by controlling the first statistical module, the first monitoring module and the first control module through the first start-stop management mechanism.

3. The real-time data acquisition task scheduling method according to claim 2, characterized in that: The first start-stop management mechanism includes: Generate a first control instruction according to the data acquired by the first statistical module and the first monitoring module; The first control module performs start and stop management according to the first control instruction; The first control instructions include control instructions for several working conditions.

4. The real-time data acquisition task scheduling method according to claim 3, characterized in that: The plurality of operating conditions at least include a first operating condition, a second operating condition and a third operating condition; When any of the first data acquisition units fails, it is recorded as the first working condition; When the number of the running first data acquisition units is greater than the required number of the first data acquisition units, it is recorded as the second working condition; When the number of the running first data acquisition units is not greater than the required number of the first data acquisition units, it is recorded as the third working condition.

5. The real-time data acquisition task scheduling method according to claim 4, characterized in that: The first data acquisition unit comprises: generating a first data acquisition logic in response to the first task scheduling; The first data collection logic is a logic designed based on the first communication module, the first parsing module and the first message processing module; The first data collection is performed by implementing the first data collection logic through the first communication module, the first parsing module and the first message processing module.

6. The real-time data acquisition task scheduling method according to claim 5, characterized in that: The first control instruction also includes: When in the first working condition, the control instruction for the first working condition includes a first screening instruction and a first allocation instruction; In the first working condition, the first data acquisition unit that fails sends a first screening instruction to the first management unit, and the first management unit performs a first screening and a second screening in response to the first screening instruction, and generates a first allocation instruction; The remaining several first data acquisition units except the faulty first data acquisition unit perform a reallocation operation according to the first allocation instruction.

7. The real-time data acquisition task scheduling method according to claim 6, characterized in that: The first data collection unit also includes: any first data collection unit is only responsible for collecting one type of business data.

8. A real-time data acquisition task scheduling system, characterized in that: include: A framework construction module, used to construct a first data acquisition framework, wherein the first data acquisition framework includes a first management unit and a plurality of first data acquisition units; The first management unit is used for first task scheduling; The first data collection unit performs first data collection in response to the first task scheduling.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of a real-time data acquisition task scheduling method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, can implement the steps of a real-time data acquisition task scheduling method as described in any one of claims 1 to 7.

Images