Data acquisition and processing method and system of intelligent measurement master station

Abstract

The invention discloses a data acquisition and processing method of an intelligent measurement master station, which relates to the technical field of intelligent measurement, and comprises the following steps: dividing a data acquisition channel, and constructing a distributed architecture of the intelligent measurement master station; cluster processing and multilink load balancing master station deployment are carried out on a distributed architecture, dynamic channel resources deployed by the master station are allocated based on an asynchronous time division multiplexing technology, and master station deployment operation based on cluster processing and multilink load balancing is carried out. According to the invention, the intelligent measurement master station system can better adapt to a complex and changeable electric power environment, the data acquisition and processing efficiency is improved, and advantages are brought to the development and application of an intelligent electric power system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent measurement, and in particular to a data acquisition and processing method and system for an intelligent measurement master station. BACKGROUND

[0002] In the current society, with the continuous development of information technology, the collection and processing of a large amount of data have become a key requirement in various industries. However, traditional data acquisition and processing methods have many problems, such as high risk of single point failure, unbalanced system load, and low resource utilization, which lead to low data processing efficiency and poor system stability. The technical solution of the present application provides an innovative solution to the problem by means of distributed architecture, load balancing, and message queue independent operation, thereby achieving efficient data acquisition and processing of the intelligent measurement master station, improving the performance and stability of the system, and meeting the social demand for big data processing.

[0003] Currently, traditional data acquisition and processing methods have many defects, such as single point failure leading to system paralysis, unbalanced load affecting system performance, and low data processing efficiency. The present application adopts distributed architecture, load balancing, and message queue independent operation, effectively solving these problems, achieving efficient operation and stability of the intelligent measurement master station, and improving the efficiency and reliability of data processing, bringing a more advanced data acquisition and processing solution to society. SUMMARY

[0004] In view of the problems existing in the prior art of data acquisition and processing method and system for an intelligent measurement master station, the present application is proposed.

[0005] Therefore, the purpose of the present application is to provide a data acquisition and processing method and system for an intelligent measurement master station, which solves the problems of single point failure and unbalanced system load in the prior art by adopting distributed architecture, load balancing, and message queue independent operation.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a data acquisition and processing method for an intelligent measurement master station, which includes dividing data acquisition channels, the division including a fee control dedicated channel and an active acquisition channel, and constructing a distributed architecture of the intelligent measurement master station; performing cluster processing and multi-link load balancing master station deployment on the distributed architecture, the master station deployment including independent operation using a message queue; and allocating dynamic channel resources of the master station deployment based on asynchronous time division multiplexing technology, and performing master station deployment operation based on cluster processing and multi-link load balancing.

[0008] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, wherein: the fee control special channel includes transmission of fee control instructions and time-of-use electricity price parameters, the active acquisition channel includes transmission of task scheduling and acquisition instruction generation data, information of fee control instructions and time-of-use electricity price parameters is transmitted through the fee control special channel, the fee control special channel includes data transmission combined with a load balancing strategy and a time-of-use multiplexing strategy, the load balancing strategy includes balancing data-based transmission load to achieve a balanced state, and the specific steps are as follows:

[0009] The data of fee control instructions and time-of-use electricity price parameters is divided into multiple data pieces and distributed to different servers or network nodes for transmission, each data piece carries part of the data, and the load of a single device or network node is reduced.

[0010] According to the real-time monitored network traffic and load conditions, the data transmission path is dynamically selected, the data transmission task is distributed to the path with lighter load, and load balancing is achieved.

[0011] A plurality of server clusters are constructed, the data transmission task is distributed to different servers for transmission, and the task distribution strategy is dynamically adjusted by monitoring the load conditions of each server in real time.

[0012] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, wherein: the time-of-use multiplexing strategy includes dynamically adjusting the load using network resources under high-frequency data transmission requirements, and the specific steps are as follows:

[0013] Each data transmission task is divided into multiple time slices, each time slice includes part of the data, and the time slices are distributed to different network connections or transmission paths to realize time-of-use multiplexing.

[0014] A data packet scheduling plan is made according to the priority of each data transmission task, the scheduling plan includes limiting the transmission rate of each time slice through a flow control mechanism and dynamically adjusting according to the actual network conditions and system load.

[0015] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, wherein: the construction includes determining whether the system is in a load condition by monitoring the load conditions of the system, and the load conditions include CPU utilization, memory utilization, network bandwidth utilization, disk IO utilization, connection number, and task queue length.

[0016] If the CPU utilization is greater than or equal to a threshold value alpha, the system is in a load condition; if the memory utilization is higher than or equal to a threshold value beta, the system has memory pressure.

[0017] If the network bandwidth utilization is greater than or equal to the threshold value γ, the system has network congestion, resulting in transmission delay or packet loss;

[0018] If the disk IO utilization is greater than or equal to the threshold value δ, the system has a disk IO bottleneck, resulting in decreased IO performance or increased response time;

[0019] If the number of connections is greater than or equal to the connection quantity threshold value ε, the system has a connection number overload, resulting in service rejection or increased response time;

[0020] If the task queue length is higher than or equal to the queue length threshold value ρ, the system has a task backlog, resulting in service delay or timeout;

[0021] At this time, the number allocated to different servers is determined according to the Least algorithm, and the specific algorithm formula is:

[0022] S i =argmin j (C j *F j )

[0023] Where F j represents the load of the server, S i represents the number of the i-th task allocated to the server, and C j represents the current number of connections of the j-th server node;

[0024] The dynamic routing is used to monitor the network traffic in real time, and the optimal transmission path is selected to reduce network congestion, and the specific calculation formula is:

[0025]

[0026] Where P i represents the priority of the i-th transmission path, W j represents the weight of the corresponding server node, F j represents the load of the server node, and D ij represents the distance or delay of the i-th transmission path to the j-th server node;

[0027] The time division multiplexing strategy also includes allocating transmission tasks of different priorities to different time slices and dynamically adjusting the transmission rate of each time slice;

[0028] If the utilization of each load is less than each threshold value, the number allocated to different servers is determined according to the load balancing algorithm at this time, and the specific algorithm formula is:

[0029] S i =(S i-1 +F i )mod N

[0030] wherein F i represents a complex function, S i represents the server number to which the i-th task is assigned, S i-1 represents the server number to which the previous task is assigned, and N represents the total number of server nodes.

[0031] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, the cluster processing includes forming a cluster by using a load balancer, and controlling the failure and load fluctuation of the master station data through the cluster processing and multi-link load balancing.

[0032] The load balancer is located at the front end of the cluster, and classifies the load situation, and the CPU utilization rate, memory utilization rate and disk IO utilization rate reflect the utilization of server resources.

[0033] If the utilization rate of a certain node exceeds the threshold value, the service performance is reduced, at this time, the node is deployed and the fault switching is realized.

[0034] The connection number and the task queue length reflect the busy degree of the system when processing the request, when the connection number or the task queue length rises, the system response becomes slow, at this time, the node is deployed and the fault switching is realized.

[0035] The network bandwidth utilization rate is distributed to different transmission paths or nodes through load balancing processing

[0036] When a certain node fails or the load rises, the load balancer is detected, at this time, the request is dynamically forwarded to the normal node.

[0037] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, the cluster processing includes forming a cluster by using a load balancer, and controlling the failure and load fluctuation of the master station data through the cluster processing and multi-link load balancing.

[0038] If the node fails, the whole system is affected, at this time, the message queue is deployed in the cluster, the message queue includes using the master-slave replication or distributed copy mechanism for fault switching, in the message queue cluster, the message data is copied to the node and ensures the consistency of the data.

[0039] When a certain node fails, the node accesses the message copy and continues to process the message.

[0040] As a preferred scheme of the data acquisition and processing method of the intelligent measurement master station, the distribution comprises adjusting resource distribution according to current system load and resource utilization, simulating group behavior among individuals in a bird group through a particle swarm algorithm, searching for an optimal solution by continuously adjusting positions and speeds of particles, and the specific calculation formula is:

[0041]

[0042] wherein, a speed of the i-th particle in the t-th generation, a position of the i-th particle in the t-th generation, a historical optimal position of the i-th particle, a global optimal position, w represents an inertia weight, c1 and c2 represent acceleration factors, and r1 and r2 represent random numbers;

[0043] The optimal solution is searched for the cluster according to the group behavior among individuals in the bird group, and the data of the intelligent measurement master station is distributed.

[0044] In a second aspect, an embodiment of the present application provides a data acquisition and processing system of an intelligent measurement master station, which comprises: a construction module that divides data acquisition channels, the division comprising a fee control special channel and an active acquisition channel, and constructs a distributed architecture of the intelligent measurement master station; a running module that carries out cluster processing and multi-link load balancing of the master station deployment of the distributed architecture, the master station deployment comprising independent running by using a message queue; and a distribution module that distributes dynamic channel resources of the master station deployment based on an asynchronous time-division multiplexing technology, and carries out master station deployment operation based on cluster processing and multi-link load balancing.

[0045] In a third aspect, an embodiment of the present application provides a computer device comprising a memory and a processor, and the memory stores a computer program, wherein the processor implements any step of the data acquisition and processing method of the intelligent measurement master station when executing the computer program.

[0046] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement any step of the data acquisition and processing method of the intelligent measurement master station.

[0047] The beneficial effects of this invention are as follows: This invention employs a distributed architecture, load balancing, and independent message queue operation technology, effectively solving many problems in traditional intelligent metering master station systems. The distributed architecture makes the system more flexible and scalable, the load balancing technology ensures the rational utilization of resources, and the independent operation of the message queue reduces the risk of single points of failure. This invention enables intelligent metering master station systems to better adapt to complex and changing power environments, improves the efficiency of data acquisition and processing, and brings advantages to the development and application of intelligent power systems. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0049] Figure 1 The following is a detailed flowchart of a data acquisition and processing method and system for an intelligent measurement master station, provided as an embodiment of the present invention.

[0050] Figure 2 This is an internal structural diagram of a computer device for a data acquisition and processing method and system for an intelligent measurement master station, provided as an embodiment of the present invention. Detailed Implementation

[0051] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0052] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0054] The application is described in detail in combination with the schematic diagram. In the detailed description of the embodiments of the application, the sectional view of the device structure is partially enlarged without the general proportion for the convenience of illustration, and the schematic diagram is only an example which should not limit the scope of protection of the application herein. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in the actual manufacture.

[0055] Meanwhile, in the description of the application, it should be noted that the terms "upper, lower, inner and outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first, second or third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0056] In the application, unless otherwise explicitly specified and limited, the terms "mounting, connection, connection" should be understood broadly, for example: it can be fixed connection, detachable connection or integral connection; it can also be mechanical connection, electrical connection or direct connection, it can also be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0057] Embodiment 1

[0058] Reference Figure 1 and Figure 2 The first embodiment of the application provides a data acquisition and processing method of an intelligent measurement master station, which comprises:

[0059] S1: Dividing the data acquisition channel, including the fee control special channel and the active acquisition channel, and constructing the distributed architecture of the intelligent measurement master station.

[0060] Among them, the fee control special channel includes the transmission of fee control instructions and time-of-use electricity price parameters, and the active acquisition channel includes the transmission of task scheduling and acquisition instruction generation data. The information of fee control instructions and time-of-use electricity price parameters is transmitted through the fee control special channel. The fee control special channel includes data transmission combined with load balancing strategy and time-sharing multiplexing strategy. The load balancing strategy includes balancing data based on balancing the load of transmission to achieve a balanced state. The specific steps are as follows:

[0061] The data of fee control instructions and time-of-use electricity price parameters is divided into multiple data pieces and distributed to different servers or network nodes for transmission. Each data piece carries part of the data, reducing the load of a single device or network node.

[0062] According to the real-time monitoring of network traffic and load, the path of data transmission is dynamically selected, and load balancing is realized by allocating data transmission tasks to the path with lighter load.

[0063] A plurality of server clusters are constructed, data transmission tasks are allocated to different servers for transmission, and the task allocation strategy is dynamically adjusted by real-time monitoring of the load of each server.

[0064] Further, the time division multiplexing strategy includes dynamically adjusting the load using network resources under high-frequency data transmission demand, and the specific steps are as follows:

[0065] Each data transmission task is divided into a plurality of time slices, each time slice includes part of the data, and the time slices are allocated to different network connections or transmission paths to realize time division multiplexing.

[0066] Based on the transmission tasks of each data, a data packet scheduling plan is made according to the priority, and the scheduling plan includes limiting the transmission rate of each time slice through a flow control mechanism and dynamically adjusting according to the actual network status and system load.

[0067] S1.1: Constructing includes determining whether it is in load condition by monitoring the load of the system, the load condition includes CPU utilization, memory utilization, network bandwidth utilization, disk IO utilization, connection number and task queue length;

[0068] If the CPU utilization is greater than or equal to the threshold value α, the system is in load condition; if the memory utilization is higher than or equal to the threshold value β, the system has memory pressure;

[0069] If the network bandwidth utilization is greater than or equal to the threshold value γ, the system has network congestion, resulting in transmission delay or packet loss;

[0070] If the disk IO utilization is greater than or equal to the threshold value δ, the system has disk IO bottleneck, resulting in IO performance degradation or response time increase;

[0071] If the connection number is greater than or equal to the connection number threshold value ε, the system has connection number overload, resulting in service rejection or response time increase;

[0072] If the task queue length is greater than or equal to the queue length threshold value ρ, the system has task backlog, resulting in service delay or timeout;

[0073] At this time, the number allocated to different servers is determined according to the Least algorithm, and the specific algorithm formula is:

[0074] S i =argmin j (C j *F j )

[0075] wherein F j represents the load condition of the server, S i represents the server number to which the ith task is assigned, C j represents the current number of connections of the jth server node;

[0076] The dynamic routing is used to monitor the network traffic in real time, and the optimal transmission path is selected to reduce network congestion. The specific calculation formula is:

[0077]

[0078] wherein P i represents the priority of the ith transmission path, W j represents the weight of the corresponding server node, F j represents the load condition of the server node, D ij represents the distance or delay of the ith transmission path to the jth server node.

[0079] The time division multiplexing strategy also includes assigning transmission tasks of different priorities to different time slices and dynamically adjusting the transmission rate of each time slice.

[0080] If the utilization rates of the load conditions are less than the respective thresholds, the number of the server to which the task is assigned is determined according to the load balancing algorithm at this time. The specific algorithm formula is:

[0081] S i = (S i-1 +F i ) mod N

[0082] wherein F i represents a complex function, S i represents the server number to which the ith task is assigned, S i-1 represents the server number to which the previous task is assigned, and N represents the total number of server nodes.

[0083] Preferably, the system monitors the CPU utilization rate, memory utilization rate, network bandwidth utilization rate, disk IO utilization rate, connection number and task queue length. When the CPU utilization rate is higher than 80%, the memory utilization rate is higher than 70%, the network bandwidth utilization rate is higher than 90%, the disk IO utilization rate is higher than 75%, the connection number exceeds 1000 and the task queue length exceeds 200, the system is in a load state. The Least algorithm is used for server assignment, the dynamic routing is used to monitor the network traffic in real time, the transmission path priority is selected, and the time division multiplexing strategy is used to ensure system load balancing. The comparison of load state data is shown in Table 1.

[0084] Table 1 Comparison of load state data

[0085] Load condition Threshold Actual value Whether in load state CPU utilization 80％ 85％ Yes Memory utilization 70％ 65％ No Network bandwidth utilization 90％ 85％ No Disk IO utilization 75％ 70％ No Connection number 1000 1200 Yes

[0086] The table compares the key indicators of the system under different load conditions, such as CPU utilization, memory utilization, and network bandwidth utilization, etc. By comparing the data, the optimal load state is determined.

[0087] S2: Cluster processing and multi-link load balancing master station deployment for distributed architecture, including independent running with message queue.

[0088] Among them, cluster processing includes forming a cluster with a load balancer, and controlling the failure and load fluctuation of the master station data through cluster processing and multi-link load balancing master station deployment;

[0089] The load balancer is located at the front end of the cluster, and classifies the load situation. CPU utilization, memory utilization, and disk IO utilization reflect the utilization of server resources;

[0090] If the utilization of a node exceeds the threshold, the service performance will decrease. At this time, the node is deployed and fault switching is realized;

[0091] The number of connections and the length of the task queue reflect the degree of busyness when the system processes requests. When the number of connections or the length of the task queue increases, the system response will slow down. At this time, the node is deployed and fault switching is realized;

[0092] Network bandwidth utilization distributes traffic to different transmission paths or nodes through load balancing processing

[0093] When a node fails or the load increases, the load balancer detects it, and at this time, the request is dynamically forwarded to the normal node.

[0094] S2.1: Independent running includes cluster formed by combining load balancer to process master station node;

[0095] If the node fails, the entire system will be affected. At this time, the message queue is deployed in the cluster, which includes using master-slave replication or distributed replica mechanism for failover. In the message queue cluster, message data will be replicated to the node and ensure data consistency;

[0096] When a node fails, the node accesses the message copy and continues message processing.

[0097] S3: Based on asynchronous time multiplexing technology, allocate dynamic channel resources for master station deployment, and carry out master station deployment operation based on cluster processing and multi-link load balancing.

[0098] The independent running includes processing of the master node by a cluster formed in combination with a load balancer.

[0099] If the node fails, the entire system is affected, at which time a message queue is deployed in the cluster, the message queue including failover using master-slave replication or distributed replica mechanism, in the message queue cluster, message data is replicated to the node and consistency of the data is ensured.

[0100] When a node fails, the node accesses the message copy and continues message processing, and the parameter and value comparison is shown in Table 2 as follows:

[0101] The parameter and value comparison is shown in Table 2 as follows:

[0102] Parameter Value Other data Node number 3 - Replication factor 2 - Network bandwidth 1 Gbps - Memory capacity 16 GB - Storage capacity 1 TB -

[0103] The selection of parameter values has been comprehensively considered, the number of nodes is set to 3 to balance the performance and cost of the system, and at the same time, a certain fault tolerance capability is provided, the replication factor is selected to be 2 to provide data redundancy backup and fault tolerance capability, and the network bandwidth, memory capacity and storage capacity are adjusted according to the demand of the system and the expected load to ensure that the system can run stably in various situations.

[0104] S3.1: allocation includes adjusting resource allocation according to current system load and resource utilization, simulating the group behavior between individuals in a bird flock through a particle swarm algorithm, searching for the optimal solution by constantly adjusting the position and speed of each particle, and the specific calculation formula is:

[0105]

[0106] wherein, represents the speed of the i-th particle in the t-th generation, represents the position of the i-th particle in the t-th generation, represents the historical optimal position of the i-th particle, represents the global optimal position, w represents the inertia weight, c1 and c2 represent the acceleration factor, and r1 and r2 represent random numbers;

[0107] According to the group behavior between individuals in a bird flock, the cluster is searched for the optimal solution, and the data of the intelligent measurement master station is distributed.

[0108] Further, the state of the cluster and the position of each particle are corresponded, usually, the position of the particle can be mapped to a certain state in the cluster, for example, the position of each particle may represent a possible task allocation scheme or data transmission path scheme, according to the position of each particle, the task allocation and data transmission path of the nodes in the cluster are adjusted accordingly to realize the optimal allocation of resources.

[0109] In a preferred embodiment, a data acquisition and processing system of an intelligent measurement master station, the system includes a construction module that divides data acquisition channels, the division including a fee control dedicated channel and an active acquisition channel, and constructs a distributed architecture of the intelligent measurement master station; a running module that carries out cluster processing and multi-link load balancing of the master station deployment of the distributed architecture, the master station deployment including independent running using a message queue; and an allocation module that allocates dynamic channel resources of the master station deployment based on an asynchronous time-division multiplexing technology, and carries out master station deployment operation based on cluster processing and multi-link load balancing.

[0110] The above-mentioned unit modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above-mentioned modules by the processor.

[0111] In an embodiment, a computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0112] In summary, the present application adopts distributed architecture, load balancing and message queue independent running technology, effectively solves many problems in the traditional intelligent measurement master station system, the distributed architecture makes the system more flexible and extensible, the load balancing technology ensures the rational use of resources, and the independent running of the message queue reduces the risk of single point failure. The present application makes the intelligent measurement master station system better adapt to the complex and changeable power environment, improves the efficiency of data acquisition and processing, and brings advantages to the development and application of the intelligent power system.

[0113] Example 2

[0114] Reference Figure 1 and Figure 2For the second embodiment of the application, the application provides a data acquisition and processing method of an intelligent measurement master station.

[0115] In the simulation environment, three virtual server nodes will be created, representing node A, node B and node C respectively. The initial resource utilization and load conditions will be set. If the CPU utilization of node A is 70%, the memory utilization is 60%, the network bandwidth utilization is 80%, the disk IO utilization is 50%, the connection number is 100, and the task queue length is 50. The resource utilization and load conditions of node B and node C are also set in a similar manner.

[0116] The particle swarm algorithm is used to dynamically allocate system resources to maximize system performance indicators such as throughput and response speed. The system load is monitored by the load balancer, and the data transmission path is adjusted according to the dynamic routing strategy to reduce network congestion and improve system stability. Message queues are deployed between server nodes to achieve fault switching and data consistency guarantee, improving the reliability and fault tolerance of the system. Through the above experimental steps, the performance of the technical solution of the application in the simulation environment will be evaluated, and its adaptability to system load conditions and performance improvement effect will be verified. The comparison data of load condition diversity is shown in Table 3 below:

[0117] Table 3 Comparison of load condition diversity data

[0118] Parameter Node A Node B Node C CPU utilization (%) 70 65 75 Memory utilization (%) 60 55 65 Network bandwidth utilization (%) 80 75 85 Disk IO utilization (%) 50 45 55 Connection number 100 90 110 Task queue length 50 40 60

[0119] The table compares the load conditions of different nodes, including CPU utilization, memory utilization, network bandwidth utilization and disk IO utilization, as well as connection number and task queue length. Through multiple specific value comparisons, the uniqueness of the specific values selected by the application in diversity is highlighted. The comparison with the prior art is shown in Table 4 below:

[0120] Table 4 Comparison with prior art

[0121] Comparison item Invention technical solution Prior art Distributed architecture Support Partial support Independently running message queue Yes No Dynamic channel resource allocation Yes No Multi-link load balancing Yes Partial support Particle swarm algorithm optimization Yes No Fault transfer and message processing Yes Partial support

[0122] The technical solution of the application has obvious advantages in distributed architecture, independently running message queue, dynamic channel resource allocation, multi-link load balancing, particle swarm algorithm optimization and fault transfer, and has more comprehensive and advanced functions compared with the prior art.

[0123] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A data acquisition and processing method for an intelligent measurement master station, characterized in that: include, The data acquisition channels are divided into a dedicated cost control channel and an active acquisition channel to construct a distributed architecture for the intelligent measurement master station. Deploying a master station for cluster processing and multi-link load balancing in a distributed architecture, wherein the master station deployment includes independent operation using message queues; The asynchronous time-division multiplexing technology is used to allocate dynamic channel resources for the main station deployment, and the main station deployment operation is carried out based on cluster processing and multi-link load balancing.

2. The data acquisition and processing method of the intelligent measurement master station as described in claim 1, characterized in that: The dedicated fee control channel includes the transmission of fee control instructions and time-of-use pricing parameters. The active data acquisition channel includes the transmission of task orchestration and data generation for acquisition instructions. The information of fee control instructions and time-of-use pricing parameters is transmitted through the dedicated fee control channel. The dedicated fee control channel includes data transmission combining load balancing and time-of-use multiplexing strategies. The load balancing strategy includes achieving a balanced load based on balancing data. The specific steps are as follows: The data of the fee control instructions and time-of-use electricity price parameters are divided into multiple data slices and distributed to different servers or network nodes for transmission. Each data slice carries part of the data, reducing the load on a single device or network node. Based on real-time monitoring of network traffic and load, the data transmission path is dynamically selected, and load balancing is achieved by distributing data transmission tasks to paths with lighter loads. Build multiple server clusters, distribute data transmission tasks to different servers, and dynamically adjust the task allocation strategy by monitoring the load of each server in real time.

3. The data acquisition and processing method of the intelligent measurement master station as described in claim 2, characterized in that: The time-sharing multiplexing strategy includes dynamically adjusting the load using network resources under high-frequency data transmission demands. The specific steps are as follows: Each data transmission task is divided into multiple time slices, each time slice including a portion of the data. Time slices are allocated to different network connections or transmission paths to achieve time-division multiplexing. Based on the data transmission tasks, a data packet scheduling plan is formulated according to priority. The scheduling plan includes limiting the transmission rate of each time slice through a flow control mechanism and making dynamic adjustments according to the actual network conditions and system load.

4. The data acquisition and processing method of the intelligent measurement master station as described in claim 3, characterized in that: The construction includes determining whether the system is under load by monitoring the load conditions of the system, including CPU utilization, memory utilization, network bandwidth utilization, disk I / O utilization, number of connections, and task queue length. If CPU utilization is greater than or equal to threshold α, the system is under load; if memory utilization is greater than or equal to threshold β, the system is under memory pressure. If the network bandwidth utilization is greater than or equal to the threshold γ, the system is congested, which leads to transmission delay or packet loss. If the disk I / O utilization is greater than or equal to the threshold δ, then the system has a disk I / O bottleneck, which leads to a decrease in I / O performance or an increase in response time. If the number of connections is greater than or equal to the connection number threshold ε, the system is overloaded with connections, which may lead to service rejection or increased response time. If the task queue length is greater than or equal to the queue length threshold ρ, the system will have a backlog of tasks, resulting in service delays or timeouts. At this point, the Least algorithm is used to determine the assigned server IDs. The specific algorithm formula is as follows: S i =argmin j (C j *F j ) Among them, F j This indicates the server load, S i C represents the server number to which the i-th task is assigned. j This represents the current number of connections on the j-th server node; Dynamic routing is used to monitor network traffic load in real time and select the optimal transmission path to reduce network congestion. The specific calculation formula is as follows: Among them, P i W represents the priority of the i-th transmission path. j F represents the weight of the corresponding server node. j D represents the load status of the server nodes. ij This represents the distance or delay from the i-th transmission path to the j-th server node; The time-division multiplexing strategy also includes allocating transmission tasks of different priorities to different time slices and dynamically adjusting the transmission rate of each time slice. If the utilization rate of each load condition is less than the respective threshold, the server number to which the load is assigned is determined according to the load balancing algorithm. The specific algorithm formula is as follows: S i =(S i-1 +F i )mod N Among them, F i S represents a complex function. i S represents the server number to which the i-th task is assigned. i-1 This indicates the server number to which the previous task was assigned, and N represents the total number of server nodes.

5. The data acquisition and processing method of the intelligent measurement master station as described in claim 4, characterized in that: The cluster processing includes forming a cluster using a load balancer, and controlling the failure and load fluctuation of the master station data through cluster processing and multi-link load balancing master station deployment. The load balancer is located at the front end of the cluster and classifies the load situation. CPU utilization, memory utilization, and disk I / O utilization reflect the utilization of server resources. If the utilization of a node exceeds the threshold, it will lead to a decrease in service performance. In this case, the node can be deployed and failover can be implemented. The number of connections and the length of the task queue reflect how busy the system is when processing requests. When the number of connections or the length of the task queue increases, the system response slows down. In this case, nodes can be deployed and failover can be implemented. Network bandwidth utilization is achieved by distributing traffic to different transmission paths or nodes through load balancing. When a node fails or its load increases, the load balancer detects this and dynamically forwards the request to a normal node.

6. The data acquisition and processing method of the intelligent measurement master station as described in claim 5, characterized in that: The independent operation includes processing the main station node by combining a cluster formed by a load balancer; If a node fails, the entire system is affected. In this case, a message queue is deployed in the cluster. The message queue includes a master-slave replication or distributed replica mechanism for failover. In the message queue cluster, message data is replicated to the nodes and data consistency is ensured. When a node fails, the node accesses a message replica and continues message processing.

7. The data acquisition and processing method of the intelligent measurement master station as described in claim 6, characterized in that: The allocation includes adjusting resource allocation based on the current system load and resource utilization. It simulates the collective behavior of individuals in a flock of birds using a particle swarm optimization algorithm, searching for the optimal solution by continuously adjusting the position and velocity of each particle. The specific calculation formula is as follows: in, This represents the velocity of the i-th particle in generation t. This represents the position of the i-th particle in generation t. This represents the historical best position of the i-th particle. The global optimal position is represented by w, the inertia weight is represented by c1 and c2, the acceleration factors are represented by r1 and r2, and the random numbers are represented by r1 and r2. The optimal solution for the flock is searched based on the group behavior among individual birds, and the data is allocated to the intelligent measurement master station.

8. A data acquisition and processing system for an intelligent measurement master station, based on the data acquisition and processing method for an intelligent measurement master station as described in any one of claims 1 to 7, characterized in that: include, The module constructs a distributed architecture for the intelligent measurement master station by dividing the data acquisition channels, including a dedicated cost control channel and an active acquisition channel. The runtime module performs cluster processing and multi-link load balancing for the main station deployment in the distributed architecture, wherein the main station deployment includes independent operation using message queues; The allocation module allocates dynamic channel resources deployed by the master station based on asynchronous time-division multiplexing technology, and carries out master station deployment operations based on cluster processing and multi-link load balancing.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data acquisition and processing method of the intelligent measurement master station according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data acquisition and processing method of the intelligent measurement master station as described in any one of claims 1 to 7.

Images