Data acquisition method and system of concentrator

Abstract

The invention provides a data acquisition method and system of a concentrator, which is applied to the technical field of data acquisition and comprises the following steps: step 1, initializing and preprocessing data acquisition; step 2, dynamically adjusting the current sampling frequency based on the data change rate; step 3, dynamically adjusting the current sampling frequency according to an anomaly detection result; 4, dynamically adjusting the current sampling frequency according to the network bandwidth; and step 5, dynamically adjusting the current sampling frequency according to cooperative driving of the multi-mode acquisition terminal. The sampling frequency is dynamically adjusted by combining a plurality of factors such as data change rate, anomaly detection, network bandwidth and cooperative work of the acquisition terminals, and the data acquisition process is optimized, so that the acquisition efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data collection, and in particular relates to a data collection method and system for a concentrator. Background Art

[0002] Data acquisition refers to the automatic collection of non-electrical or electrical signals from analog and digital units such as sensors and other devices under test, and sending them to the host computer for analysis and processing. The data acquisition system is a flexible, user-defined measurement system that combines measurement software and hardware products based on computers or other dedicated test platforms.

[0003] The data collection methods in the prior art often have the problem of fixed sampling frequency and cannot adapt to environmental changes, resulting in low data transmission efficiency, bandwidth waste and insufficient data accuracy. Therefore, how to achieve dynamic sampling frequency adjustment in IoT applications to adapt to real-time data changes and network conditions has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above problems in the prior art, the purpose of the present invention is to provide a data collection method for a concentrator, which dynamically adjusts the sampling frequency by combining multiple factors such as data change rate, anomaly detection, network bandwidth, and collaborative work of collection terminals, optimizes the data collection process, and thus improves collection efficiency and accuracy.

[0005] A data collection method for a concentrator comprises the following steps:

[0006] Step 1: Data collection initialization and preprocessing;

[0007] Step 2: Dynamically adjust the current sampling frequency based on the data change rate;

[0008] Step 3: Dynamically adjust the current sampling frequency according to the anomaly detection results;

[0009] Step 4: Dynamically adjust the current sampling frequency according to the network bandwidth;

[0010] Step 5: Dynamically adjust the current sampling frequency according to the collaborative drive of the multimodal acquisition terminal.

[0011] Preferably, step 1, data acquisition initialization and preprocessing, specifically includes the following process:

[0012] According to the initial sampling frequency f base Conduct data collection;

[0013] Preprocess the collected raw data.

[0014] Preferably, the pretreatment process specifically includes the following process:

[0015] Noise processing: Noise removal is performed through median filtering or sliding average filtering;

[0016] Fill in missing data: Missing data can be filled in through linear interpolation or nearest neighbor interpolation.

[0017] Preferably, step 2, dynamically adjusting the current sampling frequency based on the data change rate, specifically comprises the following steps:

[0018] Calculate the data point d at each moment t Compared with the data point d at the previous moment t-1 Data change rate Δd t ;

[0019] The data change rate Δd of the sampled data t With the threshold δ set high , threshold δ low Compare and dynamically adjust the current sampling frequency f according to the comparison results base To update the sampling frequency:

[0020] If Δd t >δ high , then increase the sampling frequency, and the adjusted sampling frequency is:

[0021]

[0022] If Δd t <δ lw , then reduce the sampling frequency, the adjusted sampling frequency is:

[0023]

[0024] Among them, for a certain time point t, the sampling data d of the acquisition terminal t Data change rate Δd t :

[0025]

[0026] , where Δt is the time interval.

[0027] Preferably, step 3, dynamically adjusting the current sampling frequency according to the abnormality detection result, specifically comprises the following steps:

[0028] Multiple isolated trees are constructed through the isolation forest algorithm, the sampled data is randomly divided, and the anomaly score S(x) of each data point is calculated, where H(x) is the path length of the data point, c(n) is the normalization constant;

[0029] The current sampling frequency is f newWhen the anomaly score S(x) is higher than the threshold T, the data point is judged as abnormal data. At this time, the sampling frequency is increased and the adjusted sampling frequency is f′ new =f new ×(1+S(x));

[0030] When the anomaly score S(x) is not higher than the threshold T, the data point is judged as non-abnormal data. At this time, the sampling frequency is kept unchanged and the adjusted sampling frequency is f′ new =f new .

[0031] Preferably, step 4, dynamically adjusting the current sampling frequency according to the network bandwidth, specifically comprises the following steps:

[0032] Real-time monitoring of current network bandwidth BW current , calculate the network bandwidth ratio, and dynamically adjust the current sampling frequency f′ according to the network bandwidth ratio new , the sampling frequency adjusted according to the network bandwidth ratio is:

[0033]

[0034] , where BW max is the maximum network bandwidth, and the network bandwidth ratio is

[0035]

[0036] Preferably, step 5: dynamically adjusting the sampling frequency according to the collaborative driving of the multi-modal acquisition terminal specifically includes the following steps:

[0037] Collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between the sampling data of two acquisition terminals in turn, and adjust the current sampling frequency f″ based on the correlation value new Make dynamic adjustments;

[0038] The correlation value With threshold Compare, if Then reduce the sampling frequency of one of the acquisition terminals. At this time, the sampling frequency adjusted by the acquisition terminal is like Then increase the sampling frequency of the two acquisition terminals. At this time, the sampling frequencies adjusted by the two acquisition terminals are the corresponding β is the adjustment coefficient, which ranges from 0 to 1.

[0039] The second object of the present invention is to provide a data collection system for a concentrator, which is used to implement the above-mentioned data collection method for the concentrator, comprising:

[0040] Preprocessing module: Set the initial sampling frequency f base Perform data sampling and preprocess the sampled data;

[0041] Data change rate calculation and adjustment module: used to calculate the data change rate Δd of the sampled data t , according to the data change rate Δd t With threshold δ high , threshold δ low The result of size comparison selects the corresponding sampling frequency adjustment strategy;

[0042] Anomaly detection adjustment module: calculates the anomaly score of each data point and selects the corresponding sampling frequency adjustment strategy based on the comparison between the anomaly score and the threshold T;

[0043] Network bandwidth calculation and adjustment module: used to monitor the current network bandwidth BW in real time current , calculate the network bandwidth ratio, and dynamically adjust the current sampling frequency according to the network bandwidth ratio;

[0044] Multimodal collaborative adjustment module: used to collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between the sampling data of two acquisition terminals in turn, and dynamically adjust the current sampling frequency based on the correlation value.

[0045] The third object of the present invention is to provide an electronic device, comprising: a memory and at least one processor, wherein the memory stores a computer program; the at least one processor calls the computer program in the memory so that the electronic device executes the above-mentioned concentrator data collection method.

[0046] A fourth object of the present invention is to provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data collection method of the concentrator is implemented.

[0047] The beneficial effects of the present invention are as follows: the data acquisition method and system of the concentrator dynamically adjusts the sampling frequency through the coordinated action of data change rate, anomaly detection, network bandwidth and acquisition terminal, optimizes the data acquisition process, improves the flexibility of data acquisition, and avoids bandwidth waste or data loss caused by fixed sampling frequency.

[0048] In addition, the sampling frequency is adjusted in turn by driving the data change rate, driving the anomaly detection, adjusting the network bandwidth, and adjusting the sampling frequency through the synergy of the acquisition terminals, reflecting the priority of dynamically adjusting the sampling frequency. On the basis of adjusting the sampling frequency through the data change rate, the sampling frequency is dynamically adjusted by combining the influence of anomaly detection, network bandwidth, and the synergy of the acquisition terminals on the sampling process, so that multiple influencing factors work together to improve the applicability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0051] Embodiment 1

[0052] like Figure 1 As shown, a data collection method of a concentrator comprises the following steps:

[0053] Step 1: Data collection initialization and preprocessing:

[0054] According to the initial sampling frequency f base Conduct data collection.

[0055] The collected raw data is preprocessed, and the preprocessing process specifically includes the following steps:

[0056] Noise processing: Noise removal is performed through methods such as median filtering and sliding average filtering.

[0057] Fill in missing data: Missing data can be filled in through interpolation methods such as linear interpolation and nearest neighbor interpolation.

[0058] Step 2: Dynamically adjust the current sampling frequency based on the data change rate:

[0059] Calculate the data point d at each moment t Compared with the data point d at the previous moment t-1 Data change rate Δd t ,

[0060] ,

[0061] Among them, Δt is the time interval of data collection.

[0062] The data change rate Δd of the sampled data t With the threshold δ set high, threshold δ low Compare and dynamically adjust the sampling frequency to update the sampling frequency:

[0063] If Δd t >δ high , then the sampling frequency is increased. The sampling frequency at this time is:

[0064]

[0065] If Δd t <δ low , then reduce the sampling frequency, the sampling frequency at this time is:

[0066]

[0067] Detect the severity of data changes in real time, increase the collection frequency when the data changes quickly, collect more data information, and reduce the sampling frequency when the data changes slowly to avoid wasting system resources.

[0068] Step 3: Dynamically adjust the current sampling frequency based on the anomaly detection results:

[0069] Anomaly detection is performed using the Isolation Forest algorithm. By constructing multiple isolation trees, the sampled data is randomly divided and the anomaly score S(x) of each data point is calculated. , where H(x) is the path length of the data point and c(n) is the normalization constant.

[0070] When the anomaly score S(x) is higher than the threshold T, the data point is determined to be anomaly data. At this time, the sampling frequency of the acquisition terminal where the data point is located is increased, and the current sampling frequency is f′ new =f new ×(1+S(x)) to ensure that detailed information of abnormal events is captured.

[0071] When the anomaly score S(x) is not higher than the threshold T, the data point is judged as non-abnormal data. At this time, the sampling frequency remains unchanged and the current sampling frequency is f′ new =f new .

[0072] Combining anomaly detection with dynamic sampling frequency adjustment ensures that high-frequency data can be collected in a timely manner when abnormal events occur, which can effectively improve the accuracy and real-time nature of the data.

[0073] Step 4: Dynamically adjust the current sampling frequency according to the network bandwidth:

[0074] Real-time monitoring of current network bandwidth BW current, calculate the network bandwidth ratio, dynamically adjust the sampling frequency according to the network bandwidth ratio, and the sampling frequency adjusted according to the network bandwidth ratio is:

[0075]

[0076] Among them, BW max is the maximum network bandwidth, and the network bandwidth ratio is

[0077]

[0078] Therefore, if the current network bandwidth is sufficient, that is, the network bandwidth ratio is high, increasing the sampling frequency can improve the data transmission efficiency. If the current network bandwidth is tight, that is, the network bandwidth ratio is low, reducing the sampling frequency can ensure that data transmission is not congested.

[0079] In this embodiment, when the network bandwidth ratio is set to be greater than 0.8, it indicates that the current bandwidth is sufficient, and when the network bandwidth ratio is set to be less than 0.2, it indicates that the current bandwidth is tight.

[0080] Step 5: Dynamically adjust the sampling frequency according to the collaborative drive of the multi-modal acquisition terminal:

[0081] Collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between two sampling data in turn, and dynamically adjust the sampling frequency based on the correlation value.

[0082] The correlation value With threshold Compare:

[0083] like Then reduce the sampling frequency of one of the acquisition terminals. At this time, the current sampling frequency of the acquisition terminal is

[0084] like Then increase the sampling frequency of the two acquisition terminals. At this time, the current sampling frequencies of the two acquisition terminals are the corresponding

[0085] Wherein, β is an adjustment coefficient, and its value range is between 0 and 1, which is used to control the variation range of the sampling frequency. Set to 0.9, threshold Set to 0.5.

[0086] For the correlation value Assume that there are multiple acquisition terminals S1, S2, S3, ..., S n , and their respective output sampling data are The correlation between the data of each pair of acquisition terminals is calculated according to the Pearson correlation system:

[0087]

[0088] ,in, is the data of the i-th collection terminal at time point t, is the mean of the sampling data of the collection terminal.

[0089] if This indicates that the data changes of the two acquisition terminals are highly correlated, and the sampling frequency of one of the acquisition terminals can be reduced to reduce redundancy. This indicates that the correlation between the data changes of the two acquisition terminals is low, that is, the information of the two acquisition terminals is relatively complementary. The sampling frequency of the two acquisition terminals can be increased to improve the diversity of the data.

[0090] The sampling frequency is dynamically adjusted according to the system's data change rate, anomaly detection results, network bandwidth, and the synergy between the acquisition terminals. The adjustment process complements each other and can effectively ensure that the data acquisition system works efficiently and accurately, while reducing resource waste and improving the robustness and adaptability of the system.

[0091] Embodiment 2

[0092] In a second aspect of the present invention, a data acquisition system for a concentrator is proposed, which is used to implement the data acquisition method for the concentrator described in Example 1. The data acquisition system includes a preprocessing module, a data change rate calculation adjustment module, an anomaly detection adjustment module, a network bandwidth calculation adjustment module and a multimodal collaborative adjustment module.

[0093] Among them, the preprocessing module is used to set the initial sampling frequency f base Perform data sampling and preprocess the sampled data.

[0094] The data change rate calculation and adjustment module is used to calculate the data change rate Δd of the sampled data. t , according to the data change rate Δd t With threshold δ higj , threshold δ low The result of the size comparison selects the corresponding sampling frequency adjustment strategy.

[0095] The anomaly detection adjustment module is used to calculate the anomaly score of each data point and select the corresponding sampling frequency adjustment strategy based on the result of comparing the anomaly score with the threshold T.

[0096] The network bandwidth calculation and adjustment module is used to monitor the current network bandwidth BW in real time current , calculate the network bandwidth ratio, and dynamically adjust the current sampling frequency according to the network bandwidth ratio.

[0097] The multimodal collaborative adjustment module is used to collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between the sampling data of two acquisition terminals in turn, and dynamically adjust the current sampling frequency based on the correlation value.

[0098] Embodiment 3

[0099] According to a third aspect of the present invention, an electronic device is proposed, comprising: a memory and at least one processor, wherein the memory stores a computer program; the at least one processor calls the computer program in the memory so that the electronic device executes the data collection method of the concentrator described in Example 1.

[0100] Embodiment 4

[0101] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data collection method of the concentrator described in the first embodiment is implemented.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data collection method for a concentrator, characterized in that: The steps include: Step 1: Data collection initialization and preprocessing; Step 2: Dynamically adjust the current sampling frequency based on the data change rate; Step 3: Dynamically adjust the current sampling frequency according to the anomaly detection results; Step 4: Dynamically adjust the current sampling frequency according to the network bandwidth; Step 5: Dynamically adjust the current sampling frequency according to the collaborative drive of the multimodal acquisition terminal.

2. The data collection method of the concentrator according to claim 1, characterized in that: The step 1, data collection initialization and preprocessing, specifically includes the following process: According to the initial sampling frequency f base Conduct data collection; Preprocess the collected raw data.

3. The data collection method of the concentrator according to claim 2, characterized in that: The pre-processing process specifically includes the following processes: Noise processing: Noise removal is performed through median filtering or sliding average filtering; Fill in missing data: Missing data can be filled in through linear interpolation or nearest neighbor interpolation.

4. The data collection method of the concentrator according to claim 1, characterized in that: The step 2, dynamically adjusting the current sampling frequency based on the data change rate, specifically includes the following steps: Calculate the data point d at each moment t Compared with the data point d at the previous moment t-1 Data change rate Δd t ; The data change rate Δd of the sampled data t With the threshold δ set high , threshold δ low Compare and dynamically adjust the current sampling frequency f according to the comparison results base To update the sampling frequency: Among them, for a certain time point t, the sampling data d of the acquisition terminal t Data change rate Δd t : ， Where Δt is the time interval.

5. The data collection method of the concentrator according to claim 1, characterized in that: The step 3, dynamically adjusting the current sampling frequency according to the abnormality detection result, specifically includes the following steps: Multiple isolated trees are constructed through the isolation forest algorithm, the sampled data is randomly divided, and the anomaly score S(x) of each data point is calculated, where H(x) is the path length of the data point, c(n) is the normalization constant; The current sampling frequency is f new When the anomaly score S(x) is higher than the threshold T, the data point is judged as abnormal data. At this time, the sampling frequency is increased and the adjusted sampling frequency is f n ′ ew =f new ×(1+S(x)); When the anomaly score S(x) is not higher than the threshold T, the data point is judged as non-abnormal data. At this time, the sampling frequency remains unchanged and the adjusted sampling frequency is f n ′ ew =f new .

6. The data collection method of the concentrator according to claim 1, characterized in that: The step 4, dynamically adjusting the current sampling frequency according to the network bandwidth, specifically includes the following steps: Real-time monitoring of current network bandwidth BW current , calculate the network bandwidth ratio, and dynamically adjust the current sampling frequency f according to the network bandwidth ratio n ′ ew , the sampling frequency adjusted according to the network bandwidth ratio is: ， Among them, BW max is the maximum network bandwidth, and the network bandwidth ratio is 7. The data collection method of the concentrator according to claim 1, characterized in that: The step 5: dynamically adjusting the sampling frequency according to the collaborative driving of the multi-modal acquisition terminal, specifically includes the following steps: Collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between the sampling data of two acquisition terminals in turn, and adjust the current sampling frequency f″ based on the correlation value new Make dynamic adjustments; The correlation value With threshold Compare, if Then reduce the sampling frequency of one of the acquisition terminals. At this time, the sampling frequency adjusted by the acquisition terminal is like Then increase the sampling frequency of the two acquisition terminals. At this time, the sampling frequencies adjusted by the two acquisition terminals are the corresponding β is the adjustment coefficient, which ranges from 0 to 1.

8. A data acquisition system for a concentrator, characterized in that: A data collection method for implementing a concentrator according to any one of claims 1 to 7, comprising: Preprocessing module: Set the initial sampling frequency f base Perform data sampling and preprocess the sampled data; Data change rate calculation and adjustment module: used to calculate the data change rate Δd of the sampled data t , according to the data change rate Δd t With threshold δ high , threshold δ low The result of size comparison selects the corresponding sampling frequency adjustment strategy; Anomaly detection adjustment module: calculates the anomaly score of each data point and selects the corresponding sampling frequency adjustment strategy based on the comparison between the anomaly score and the threshold T; Network bandwidth calculation and adjustment module: used to monitor the current network bandwidth BW in real time current , calculate the network bandwidth ratio, and dynamically adjust the current sampling frequency according to the network bandwidth ratio; Multimodal collaborative adjustment module: used to collaboratively analyze the sampling data of multiple acquisition terminals, calculate the correlation between the sampling data of two acquisition terminals in turn, and dynamically adjust the current sampling frequency based on the correlation value.

9. An electronic device, characterized in that: The electronic device comprises: a memory and at least one processor, wherein the memory stores a computer program; the at least one processor calls the computer program in the memory so that the electronic device executes the data collection method of the concentrator as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data collection method of the concentrator according to any one of claims 1 to 7 is implemented.

Images