Elastic scalable industrial data acquisition system and method based on rule scheduling

By introducing an automatic scaling mechanism based on rule scheduling in the industrial data acquisition system, the problem of existing systems requiring manual intervention when business capacity changes is solved, and the effect of automatically adapting to business changes, reducing costs and improving system stability is achieved.

CN114745606BActive Publication Date: 2025-03-14JIANGSU SUYUN INFORMATION TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210166273.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-23
Publication Date
2025-03-14
Estimated Expiration
2042-02-23

AI Technical Summary

Technical Problem

Existing industrial data acquisition systems require manual evaluation and manual deployment of nodes when business capacity changes, resulting in increased costs, unstable operation, waste of resources and risk of system downtime.

Method used

The elastic scalable industrial data acquisition system based on rule scheduling is adopted to monitor resource usage through traffic observation components. The rule engine automatically schedules the container orchestration engine according to the scheduling rules to realize automatic scaling of nodes.

Benefits of technology

It can automatically adapt to business changes without manual intervention, reduce implementation costs, improve system stability and resource utilization, and avoid system downtime and resource waste.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114745606B_ABST
    Figure CN114745606B_ABST
Patent Text Reader

Abstract

The present invention provides an elastically scalable industrial data acquisition system based on rule scheduling, including a database, multiple online message access and processing nodes, a rule base, a traffic observation component, a rule engine, and a container orchestration engine; the rule engine is connected to the rule base and the container orchestration engine respectively, and the rule engine schedules the overall system resources through the container orchestration engine based on the scheduling rules in the rule base, without manual intervention and additional deployment and implementation, thereby reducing the implementation cost and running stably and reliably, with high implementation efficiency and good robustness. The present invention also provides an industrial data acquisition method with corresponding advantages.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial data collection, and in particular relates to an elastically scalable industrial data collection system based on rule scheduling and a corresponding industrial data collection method. Background Art

[0002] Industrial data collection is widely used in real-time data collection of various equipment and sensors in industrial sites. It is the basis for material tracking, production planning, product history maintenance and other production management. It plays a vital role in production decision-making, process optimization, quality improvement and other aspects of modern enterprises. Figure 1 In the general case of the example, multiple field devices are connected to the industrial data acquisition system through the industrial gateway. The industrial gateway sends the collected data to the industrial data acquisition system through MQTT messages. The message access and processing nodes in the industrial data acquisition system are responsible for message protocol parsing, data conversion and processing, business event generation, data storage, etc. The message access and processing nodes run multiple threads, and the operation of the threads consumes storage resources, computing resources, and power.

[0003] In the current mainstream industrial data acquisition technology, as more and more field devices and industrial gateways are added to the industrial data acquisition system, the limited number of message access and processing nodes in the system gradually become the system bottleneck. In the existing practice, in order to ensure the normal operation of the business over a period of time, it is usually necessary to do capacity planning and forecasting before the system is implemented. When the business capacity exceeds the previous plan, the operation and maintenance or implementation personnel need to re-evaluate the number of nodes, and then manually increase the message access and processing nodes to cope with the increase in business capacity. When the number of devices decreases and the message traffic decreases, a certain number of message access and processing nodes need to be manually destroyed to reduce resource consumption.

[0004] It can be seen that the existing industrial data collection still has many defects in practical applications, such as: 1. The business changes of enterprises are often not smooth. When the business suddenly increases or decreases, it requires manual intervention of management, operation and maintenance, and implementation personnel to conduct corresponding evaluations and manual deployment operations, which is equivalent to re-implementing a small project, increasing costs and even delaying the normal delivery of new businesses. 2. Manual changes to the existing architecture are often prone to errors and failures due to human operations, which have a great impact on the normal and stable operation of the business. 3. If the message traffic continues to increase without adding business nodes in time, the system will crash due to the processing capacity of the existing nodes reaching its limit. 4. If the newly added business capacity is reduced due to business changes, the extra nodes will cause a waste of resources and costs, unless manual intervention is made again to reduce the number of nodes accordingly.

[0005] Therefore, there is a great need to study an industrial data collection system and corresponding methods that can be used to implement industrial data collection in line with the increase or decrease of business, achieve cost reduction and efficiency improvement, break through the bottleneck of existing industrial data collection technology in enterprise applications, so as to further promote the in-depth development and high-efficiency application of industrial data collection technology. Summary of the invention

[0006] The present invention is to solve all or part of the problems of the above-mentioned prior art. On one hand, the present invention provides an elastically scalable industrial data collection system based on rule scheduling, which can automatically scale up and down for industrial data collection. Another aspect of the present invention provides an elastically scalable industrial data collection method based on rule scheduling, which can collect industrial data without manual work and adapt to the increase or decrease of business.

[0007] On one hand, the present invention provides an elastically scalable industrial data acquisition system based on rule scheduling, including a database, multiple online message access and processing nodes; also including: a rule base, a traffic observation component, a rule engine, and a container orchestration engine; the online message access and processing node is connected to the traffic observation component to report current processing information; the traffic observation component is connected to the rule engine to collect and summarize the current processing information of the online message access and processing node and deliver it to the rule engine; the rule engine is respectively connected to the rule base and the container orchestration engine, and the rule engine schedules the overall system resources through the container orchestration engine based on the scheduling rules in the rule base. By setting the message observation component, it is possible to monitor changes in resource usage in the industrial data acquisition system; through the rule base and the rule engine, it is possible to determine in real time whether the current industrial data acquisition system triggers the scheduling rules based on the current resource usage of the industrial data acquisition system summarized and fed back by the message observation component, based on the scheduling rules in the rule base; when any scheduling rule is triggered, the rule engine can automatically schedule the container orchestration engine to schedule system resources in a timely manner without manually changing the existing operating architecture, avoiding errors and failures caused by human operations, ensuring the normal and stable operation of the business, and requiring no additional manual real-time deployment when the business volume suddenly changes. The industrial data acquisition system has low application cost and high operating efficiency.

[0008] The industrial data acquisition system also includes: multiple standby message access and processing nodes, registration and traffic distribution components; the container orchestration engine is connected to the standby message access and processing nodes, and the standby message access and processing nodes are deployed online; when deployed online, the standby message access and processing nodes establish a connection with the registration and traffic distribution components, register through the registration and traffic distribution components, and obtain traffic distribution.

[0009] By setting up multiple backup message access and processing nodes, when the message traffic continues to increase to a certain level, the container orchestration engine can deploy the backup message access and processing nodes to go online, and the registration and traffic allocation component can register the newly deployed nodes and automatically allocate message traffic to them, so as to timely meet the resource requirements of the industrial data acquisition system and effectively avoid system downtime.

[0010] The traffic observation component includes an acquisition unit and a reporting unit; the acquisition unit is used to collect the current processing information of the online message access and processing nodes; the reporting unit is used to summarize the current processing information of all the online message access and processing nodes and deliver it to the rule engine.

[0011] The traffic observation component also includes a timing unit connected to the reporting unit, and the timing unit is used to set the reporting time or reporting cycle; the reporting unit is controlled by the timing unit to summarize the current processing information of all online message access and processing nodes according to the reporting time or reporting cycle and deliver it to the rule engine. The timing unit can preset the reporting cycle or the specific time point of reporting in accordance with the actual situation in industrial manufacturing, avoid unnecessary frequent communications, and help further optimize system resource configuration.

[0012] There are multiple scheduling rules in the rule base, and priorities are set; the rule engine includes a parsing layer and a matching trigger layer, the parsing layer receives all the online message access and current processing information of the processing node summarized by the reporting unit and parses to obtain feature data; the matching trigger layer obtains all the scheduling rules and sorts them according to the priority and matches them with the feature data. The scheduling rules in the rule base can be executed according to the priority, more flexibly matching various actual business characteristics, and more applicable.

[0013] On the other hand, the present invention provides an elastically scalable industrial data acquisition method based on rule scheduling, including: establishing a rule base to store a number of scheduling rules; observing and collecting the current processing information of all or part of the message access and processing nodes in the industrial data acquisition system, and delivering it to the rule engine; the rule engine obtains the several scheduling rules, and triggers the addition or reduction of message access and processing nodes based on the several scheduling rules. By observing the message access and processing nodes in the industrial data acquisition system and collecting the current processing information, the rule engine can obtain and summarize the business capacity and resource usage carried by the current industrial data acquisition system; by establishing a rule base and storing a number of scheduling rules, the rule engine can obtain the scheduling rules, automatically trigger the system scaling action according to the scheduling rules, and add or reduce the number of currently online message access and processing nodes in the industrial data acquisition system to match the current business capacity. The practice of relying on manual intervention evaluation and manual deployment operations to make the industrial data acquisition system match the current business needs is changed, effectively reducing or eliminating the re-implementation and deployment of manpower, reducing the overall implementation cost, optimizing the stability and fluency of the operation of the industrial data acquisition system, and further improving the application benefits of the industrial data acquisition system.

[0014] The current processing information of the message access and processing node includes: current message flow value, node CPU occupancy rate, node memory usage rate, network bandwidth occupancy rate and I / O interface usage rate. The current processing information is used as the current performance index of the evaluation system to most directly reflect the system resource utilization rate.

[0015] The basis of the several scheduling rules includes several preset thresholds; the triggering of adding or reducing message access and processing nodes based on the several scheduling rules includes: calculating and comparing the message flow values, CPU single-core occupancy rate, and memory usage rate of all the message access and processing nodes, and correspondingly obtaining the average message flow value, the maximum message flow value, the minimum message flow value, the average CPU single-core occupancy rate, and the average memory usage rate of a single message access and processing node. The average message flow value, the maximum message flow value and the minimum message flow value of a single message access and processing node, the average CPU single-core occupancy rate, and the average memory usage rate are compared with several preset thresholds to determine whether the scheduling rules are triggered, and whether to trigger adding or reducing message access and processing nodes.

[0016] There are multiple scheduling rules, which are set according to priority; if the scheduling rules are triggered, they are executed according to the priority order. The characteristics of industrial data are not single, and the resources involved in industrial data are also diverse. Multiple scheduling rules are used to deal with various situations, and priorities are set for multiple scheduling rules. When more than one scheduling rule is triggered, the scheduling rules are executed according to the priority order according to the occupation of resources by specific businesses. The rule engine can scale system resources more accurately.

[0017] The method of adding or reducing message access and processing nodes includes: the rule engine schedules the container orchestration engine to create or destroy message access and processing nodes according to the scheduling rule of adding new nodes or reducing nodes that is triggered. After any scheduling rule is triggered, the rule engine automatically schedules the container orchestration engine to automatically scale the nodes based on the triggered scheduling rule, and accordingly creates new message access and processing nodes or destroys online message access and processing nodes in the current industrial data acquisition system. The automatic and timely execution of the container orchestration engine avoids the risk of system downtime due to insufficient online message access and processing nodes when the message traffic is too high, and also avoids the waste of system resources caused by the existence of redundant online message access and processing nodes when the message traffic is too low.

[0018] The process of creating a new message access and processing node by the container orchestration engine includes: deploying a new message access and processing node; configuring environment information for the new message access and processing node; connecting the new message access and processing node to the MQTT server and database after startup, registering and establishing a heartbeat; and allocating message traffic to the new message access and processing node.

[0019] By registering the newly deployed message access and processing nodes and automatically allocating message traffic to them, the load of each message access and processing node in the current industrial data acquisition system can be dispersed and balanced.

[0020] The process of destroying message access and processing nodes by the container orchestration engine includes: deploying and destroying redundant message access and processing nodes; after the heartbeat times out, sensing that the redundant message access and processing nodes are offline; and allocating the message traffic of the redundant message access and processing nodes to the online message access and processing nodes. After individual message access and processing nodes are recycled, the message traffic of the message access and processing nodes can be automatically allocated to other online message access and processing nodes to ensure the continuous stability of data collection information traffic and timely adjustment of resource allocation.

[0021] After the message traffic is redistributed, all online message access and processing nodes report the current processing information and summarize it before delivering it to the rule engine. Any changes in message traffic are fed back to the rule engine in real time, which is conducive to timely scaling of system resources and maintaining the continuous stability of industrial data collection work.

[0022] Compared with the prior art, the main beneficial effects of the present invention are:

[0023] 1. The elastically scalable industrial data acquisition system based on rule scheduling of the present invention can monitor the changes in resource usage in the industrial data acquisition system by setting the message observation component; through the rule library and the rule engine, the current resource usage of the industrial data acquisition system summarized and fed back by the message observation component can be obtained, and the traffic can be adaptively allocated to meet the resource needs of the system in a timely manner. The system has high utilization efficiency, strong operational stability and high reliability; it is conducive to the further extensive and in-depth application of industrial data acquisition systems in intelligent manufacturing.

[0024] 2. The elastically scalable industrial data collection method based on rule scheduling of the present invention has simple steps. By storing scheduling rules in a rule base and observing and collecting current processing information, the current system resource usage is obtained. The rule engine can automatically trigger the system scaling action in combination with the scheduling rules, which changes the traditional practice of relying on manpower for re-implementation and deployment, greatly reduces the overall implementation cost, and scales system resources more accurately and timely, which effectively avoids the risk of system downtime and avoids excessive resource waste of online message access and processing nodes, improves the implementation efficiency and application benefits of the industrial data collection system, and provides a positive and effective solution for greatly improving the level of industrial intelligent manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the existing industrial data acquisition system and its application scenarios.

[0026] Figure 2 The following is a schematic diagram of an industrial data acquisition system and its application scenario according to Embodiment 1 of the present invention.

[0027] Figure 3 This is a schematic diagram of an elastically scalable industrial data collection method based on rule scheduling according to Embodiment 1 of the present invention.

[0028] Figure 4 The present invention is a flowchart of an elastically scalable industrial data collection method based on rule scheduling according to the second embodiment of the present invention.

[0029] Figure 5 The present invention is a flowchart of an elastically scalable industrial data collection method based on rule scheduling according to the third embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solutions in the specific embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the following embodiments, operations of the embodiments are described in specific sequences. The description of these sequences is for better understanding of the details in the embodiments and for a comprehensive understanding of the present invention. However, the description of these sequences does not limit the scope of the present invention.

[0032] Embodiment 1

[0033] In the first embodiment of the present invention, Figure 2 As shown, the field equipment is connected to the industrial data acquisition system through the industrial gateway, and the industrial gateway sends the collected data to the industrial data acquisition system through MQTT messages. The online message access and processing nodes in the industrial data acquisition system are responsible for message protocol parsing, data conversion and processing, business event generation, data storage, etc. In this embodiment, the industrial data acquisition system includes a database, multiple online message access and processing nodes (in order to simplify the drawings, Figure 2 In the example, there are currently two online message access and processing nodes, and the number of actual online message access and processing nodes is not limited). The example industrial data acquisition system also includes: a rule base, a traffic observation component, a rule engine, and a container orchestration engine. The online message access and processing node is connected to the traffic observation component to report the current processing information. The traffic observation component is connected to the rule engine, collects and summarizes the current processing information of the online message access and processing node and delivers it to the rule engine. The rule engine is connected to the rule base and the container orchestration engine respectively. The rule engine schedules the overall system resources through the container orchestration engine based on the scheduling rules in the rule base. The example industrial data acquisition system also includes: two standby message access and processing nodes, and a registration and traffic distribution component. The container orchestration engine is connected to the standby message access and processing node, and the standby message access and processing node is deployed online. When deployed online, the standby message access and processing node establishes a connection with the registration and traffic distribution component, registers through the registration and traffic distribution component, and obtains traffic distribution. Also, for the sake of simplicity of the attached figure, only two standby message access and processing nodes are exemplified, and the specific number in the actual situation is not limited.

[0034] The example traffic observation component includes an acquisition unit and a reporting unit. The acquisition unit is used to collect the current processing information of the online message access and processing nodes; the reporting unit is used to summarize the current processing information of all online message access and processing nodes and deliver it to the rule engine. In some embodiments, the traffic observation component also includes a timing unit connected to the reporting unit, the timing unit is used to set the reporting time or reporting cycle; the reporting unit is controlled by the timing unit to summarize the current processing information of all online message access and processing nodes according to the reporting time or reporting cycle and deliver it to the rule engine.

[0035] There are multiple scheduling rules in the rule base, and they are set with priorities. The rule engine includes a parsing layer and a matching trigger layer. The parsing layer receives all online message access and current processing information of the processing node summarized by the reporting unit and parses to obtain feature data; the matching trigger layer obtains all scheduling rules and sorts them by priority and matches them with the feature data.

[0036] In this embodiment, the elastic scaling industrial data collection method based on rule scheduling is as follows: Figure 3 As shown, it includes: establishing a rule base to store several scheduling rules; observing and collecting the current resource information and message traffic of all or part of the message access and processing nodes in the industrial data acquisition system, and delivering them to the rule engine; the rule engine obtains several scheduling rules, and triggers the addition or reduction of message access and processing nodes based on several scheduling rules. In the example, all online message access and processing nodes in the industrial data acquisition system are observed and collected. In some implementation situations, the traffic observation component observes and collects several specific online message access and processing nodes. According to the implementation business characteristics and industrial manufacturing message characteristics combined with the actual needs of the enterprise, specific online message access and processing nodes can be observed and collected, and the specific situation is not limited here.

[0037] The basis of several scheduling rules in the example includes several preset thresholds. Triggering the addition or reduction of message access and processing nodes based on several scheduling rules includes: calculating and comparing the message traffic values, CPU single-core occupancy rates, and memory usage rates of all message access and processing nodes, and correspondingly obtaining the average message traffic value, the maximum message traffic value of a single message access and processing node, the minimum message traffic value, the average CPU single-core occupancy rate, and the average memory usage rate. By comparing the preset thresholds, the scheduling rules can be triggered when the message traffic increases to a certain level, and the container orchestration engine is automatically deployed based on the triggered scheduling rules to handle the newly added business; and when the message traffic drops to a certain level, the scheduling rules are triggered to schedule the container orchestration engine to automatically remove redundant message access processing nodes, so as to achieve the purpose of reasonably reducing costs and increasing efficiency. In some embodiments, the basis of several scheduling rules may include preset node CPU occupancy thresholds and node memory usage thresholds.

[0038] The current processing information of the example includes the message traffic value of the message access and processing node, the node CPU occupancy rate, the node memory usage rate, the network bandwidth occupancy rate, and the I / O interface usage rate. For example: "If the message traffic value of a single online message access and processing node exceeds 150,000 / second and the average CPU single core occupancy rate exceeds 90%, a new message access and processing node is deployed"; "If the average message traffic value of all online message access and processing nodes exceeds 120,000 / second and the average memory usage rate of all online message access and processing nodes exceeds 80%, a new access and processing node is deployed"; "If the average message traffic value of all message access and processing nodes is less than 70,000 / second, a message access and processing node is destroyed", etc. The scheduling rules in the rule base are set with priorities. If the scheduling rules are triggered, they are executed in order of priority. Enterprises can set scheduling rules and scheduling rule priorities according to actual business conditions. When any scheduling rule is triggered, the container orchestration engine is called according to the rule to add a new message access and processing node. In some implementations, the industrial data collection system does not have spare message access and processing nodes. When the message flow is reduced to a certain level, the rule engine destroys a corresponding number of online message access and processing nodes based on the rule to adjust the number of online message access and processing nodes in a timely manner and save system resources. The specific situation is not limited here.

[0039] Embodiment 2

[0040] This embodiment provides an example of the elastic scaling industrial data collection method based on rule scheduling after the enterprise has new equipment and sensors connected. Figure 2 ,like Figure 4As shown, each online message access and processing node reports the current processing information to the traffic observation component. In the example, the traffic observation component periodically summarizes the information reported by all online message access and processing nodes and delivers it to the rule engine. The rule engine determines whether to trigger the scheduling rule based on the scheduling rules in the rule base. If the current situation does not trigger the addition of online message access and processing nodes, the traffic observation component continues to observe and collect. If the scheduling rule that requires the addition of online message access and processing nodes is triggered, the rule engine schedules the container orchestration engine to create a new online message access and processing node. The container orchestration engine deploys the standby message access and processing node to create a newly added online message access and processing node, and configures the environmental information. The container orchestration engine automatically configures the environmental information of the newly deployed node, such as the connection information of the MQTT server node, the connection information of the database node, the connection information of the traffic observation component, and the registration and traffic distribution component connection information. In this embodiment, the registration and traffic distribution component is used to register the newly deployed message access and processing node, and allocates a certain message traffic from the message traffic pool to the new node, that is, allocates a part of the message subscription topic to the newly launched message access and processing node to obtain message traffic. After the newly launched message access and processing node is started, it automatically connects to the MQTT server and database, establishes a heartbeat connection with the registration and traffic allocation component, and registers. The registration and traffic allocation component allocates traffic to the newly launched message access and processing node. The new online message access and processing node obtains the message topic for traffic allocation and subscribes. The new online message access and processing node participates in receiving message traffic and reports to the traffic observation component. If the message traffic increases again and triggers the scheduling rule again, the industrial data acquisition system continues to run, and the above process is repeated to continue to add online message access and processing nodes.

[0041] The industrial data collection system of this embodiment can trigger rules when the message traffic increases to a certain level. The rule engine schedules the container orchestration engine to automatically deploy new message access processing nodes to continuously process new services.

[0042] Embodiment 3

[0043] This embodiment provides an example of the elastic scaling industrial data collection method based on rule scheduling after the actual investment of equipment and sensors is reduced, resulting in a reduction in message traffic. Figure 5As shown, each online message access and processing node reports its current processing information to the traffic observation component. The traffic observation component periodically summarizes the information reported by all online message access and processing nodes and delivers it to the rule engine. It should be noted that the periodic summary of the traffic observation component is a good practice for the example. In some implementations, the traffic observation component can also summarize the current information in real time and deliver it to the rule engine, and the specific practice is not limited. The rule engine determines whether to trigger the scheduling rule based on the scheduling rules in the rule base. If no scheduling rule is triggered, the traffic observation component continues to collect and summarize information; if the scheduling rule that requires reducing the online message access and processing nodes is triggered, the rule engine schedules the container orchestration engine to destroy one or several online message access and processing nodes. The container orchestration engine destroys the redundant online message access and processing nodes. The registration and traffic distribution component senses that the destroyed online message access and processing node has gone offline after the heartbeat timeout. The registration and traffic distribution component distributes the message traffic of the destroyed offline message access and processing node to other current online message access and processing nodes. After the traffic is redistributed, the current online message receiving and processing node receives and processes the traffic and reports it to the traffic observation component.

[0044] The industrial data acquisition system of this embodiment can trigger the scheduling rules when the message traffic drops to a certain level, and the scheduling container orchestration engine automatically removes redundant online message access processing nodes to achieve the purpose of reducing costs and increasing efficiency.

[0045] Through the above-mentioned embodiments 2 and 3, it can be understood that the elastically scalable industrial data collection method based on rule scheduling observes and summarizes the message traffic, and automatically triggers the scheduling rules of the rule engine when the message traffic grows to a certain level, and automatically schedules the container orchestration engine to deploy new message access and processing nodes, and the new message access and processing nodes automatically obtain the allocated traffic for job processing; when the traffic drops to a certain level, the rules are automatically triggered, and the container orchestration engine is automatically scheduled to destroy the node, and the message traffic of the destroyed node is automatically redistributed. Through automatic collection, feedback, rule triggering, resource scheduling, traffic allocation and other links, it is possible to effectively reduce or eliminate the re-implementation and deployment of manpower in the above-mentioned scenarios, and reduce the overall implementation cost; the rule engine can scale system resources more accurately and timely, avoid the risk of system downtime due to excessively high message traffic and too few nodes, and avoid the waste of resources due to too low message traffic and too many nodes, effectively improving the resource utilization efficiency and continuous stability of the system.

[0046] It is apparent to those skilled in the art that several improvements and modifications may be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An elastic and scalable industrial data collection method based on rule scheduling, characterized by: include: Establish a rule base to store several scheduling rules; Observe and collect the current processing information of all or part of the message access and processing nodes in the industrial data acquisition system, and deliver it to the rule engine; The rule engine obtains the plurality of scheduling rules, and triggers adding or reducing message access and processing nodes based on the plurality of scheduling rules; The method of adding or reducing message access and processing nodes includes: the rule engine schedules the container orchestration engine to create or destroy message access and processing nodes according to the scheduling rule of adding a node or reducing a node that is triggered; The process of creating a new message access and processing node by the container orchestration engine includes: deploying a new message access and processing node; configuring environment information for the new message access and processing node; connecting the new message access and processing node to the MQTT server and database after startup, registering and establishing a heartbeat; allocating message traffic to the new message access and processing node; Deploy and destroy redundant message access and processing nodes; after the heartbeat times out, sense that the redundant message access and processing nodes are offline; allocate message traffic of the redundant message access and processing nodes to online message access and processing nodes; After the message traffic is redistributed, all online message access and processing nodes report the current processing information and summarize it before delivering it to the rule engine; when the summarized message traffic grows to the trigger condition of the rule engine, the rule engine scheduling rules automatically adjust the container orchestration engine to deploy new message access and processing nodes, and the new message access and processing nodes automatically obtain the allocated traffic for job processing; when the summarized traffic drops to the trigger condition of the rule engine, the rule engine scheduling rules automatically schedule the container orchestration engine to destroy the node, and automatically redistribute the message traffic of the destroyed node.

2. The elastic and scalable industrial data collection method based on rule scheduling according to claim 1 is characterized in that: The current processing information of the message access and processing node includes: current message flow value, node CPU occupancy rate, node memory usage rate, network bandwidth occupancy rate and I / O interface usage rate.

3. The elastic and scalable industrial data collection method based on rule scheduling according to claim 2 is characterized in that: The basis of the plurality of scheduling rules includes a plurality of preset thresholds; the triggering of adding or reducing message access and processing nodes based on the plurality of scheduling rules includes: The message flow values, CPU single-core occupancy rates, and memory utilization rates of all the message access and processing nodes are calculated and compared, and the average message flow value, the maximum message flow value, the minimum message flow value, the average CPU single-core occupancy rate, and the average memory utilization rate of a single message access and processing node are obtained accordingly.

4. The elastic and scalable industrial data collection method based on rule scheduling according to claim 1 is characterized in that: There are multiple scheduling rules, which are set according to priority; if the scheduling rules are triggered, they are executed according to the priority order.

Citation Information

Patent Citations

  • MapReduce short job optimization system and method based on resource reuse

    CN105808334A