Physical environment lightweight data acquisition method

Abstract

The invention relates to the field of environment information, and discloses a physical environment lightweight data acquisition method, which automatically triggers physical environment data acquisition by setting a preset time interval and delay response time, detects an abnormal condition during acquisition, and monitors the change of a data stream during data acquisition in real time, so as to realize the lightweight data acquisition of a physical environment. According to the method, the deviation degree of data acquisition is preliminarily analyzed to detect whether acquisition errors and gene error detection results exist in data acquisition or not, abnormal values of physical environment data are detected and fed back, then the data are subjected to lightweight processing, and the data transmission process of the physical environment data subjected to lightweight processing is analyzed, so that the data transmission accuracy is improved. The data transmission process is monitored and optimized in real time, storage of physical environment data is optimized based on a result obtained after data transmission optimization, data acquisition faults can be quickly responded and eliminated, various parameters of the physical environment can be quickly and accurately acquired, the data transmission and storage process is optimized, and the data maintenance cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of environmental information technology, and more specifically to a physical environment lightweight data collection method. Background Art

[0002] In many physical environment monitoring scenarios, especially in remote or mobile environments, data acquisition equipment often faces resource constraints, such as limited power, computing power, and bandwidth, and these constraints require data acquisition methods to be efficient and resource-saving. Due to the high deployment and maintenance costs of data acquisition equipment, especially in large-scale or long-term monitoring projects, the use of IoT technology and the combination of data from multiple sensors for data fusion and processing, and then lightweight data collection can reduce costs and improve economic benefits. However, the above process still has the following disadvantages: First, the existing lightweight data collection methods lack the delay detection and analysis process of setting up data collection, and cannot detect abnormal problems that occur during the data collection process in a timely manner, thereby reducing the efficiency of data collection and being unfavorable for rapid response and troubleshooting of data collection; Second, the existing lightweight data collection methods lack real-time monitoring of the changes in the data flow during data collection, and are unable to promptly discover the errors generated during the data collection process, resulting in an inability to understand the distribution of abnormal values ​​in the collected data, and thus unable to automatically determine the type of errors generated by data collection; Third, the existing lightweight data collection methods lack automatic optimization analysis of data transmission and storage, and are unable to optimize and adjust data transmission and storage based on the analysis results, resulting in increased data maintenance costs. Summary of the invention

[0003] In order to overcome the above-mentioned defects of the prior art, the present invention provides a lightweight data collection method for a physical environment to solve the problems existing in the above-mentioned background technology.

[0004] The present invention provides the following technical solution: a method for collecting lightweight data of a physical environment, comprising: S1: Automatically triggering physical environment data collection by setting a preset time interval on the data collection device and setting a delayed response time for data collection; S2: By reading the collected physical environment data from the data collection device and monitoring the data flow changes during data collection in real time, the deviation degree of data collection is preliminarily analyzed to obtain the collection deviation degree coefficient, which is used to detect whether there is collection error in data collection; S3: Based on the analysis results of data collection errors, detect abnormal values ​​of physical environment data and provide feedback on the detection results; S4: used to pre-process the collected physical environment data and perform lightweight processing on the pre-processed physical environment data; S5: Optimize and analyze the data transmission process based on the lightweight physical environment data, and calculate the network congestion control coefficient for real-time monitoring and optimization of the data transmission process; S6: Based on the results of data transmission optimization, the storage of physical environment data is optimized and analyzed to obtain a storage optimization index, thereby optimizing data storage in real time.

[0005] Preferably, the S1 is used to automatically trigger the collection of physical environment data and detect abnormal delays during data collection. The data collection device selects a multi-channel sensor according to the measurement object to collect physical environment data in parallel, and based on the collection frequency requirements of the data collection device, uses an embedded system to configure a timer, sets a specific time interval in the timer, and triggers an interrupt service routine when the timer overflows, thereby performing timed collection of physical environment data, recording the trigger start time and end time of each data collection, and calculating the actual time of each data collection based on the trigger start time and end time of the data collection. The specific calculation formula is: , by taking the actual time of each data collection With delayed response time Comparison is used to detect whether there are abnormal problems in data collection. When the system fails to complete the data collection within the preset delay time, it will immediately enter the warning state and send a warning reminder of data collection delay to the management personnel, thereby eliminating the abnormal problem of data collection. When , it indicates that there is no abnormal problem in data collection caused by delay, and the deviation of data collection is further analyzed.

[0006] Preferably, the S2 reads the data stream from the data acquisition device in real time, records the timestamp of the data acquisition, and then performs preprocessing operations on the read physical environment data. The specific operation process includes data cleaning, data conversion and data smoothing. By using statistical methods to analyze the change trend of the data stream within a period of acquisition time, and calculating the acquisition deviation degree coefficient, the acquisition deviation degree coefficient is used to preliminarily detect whether there is an acquisition error in the data acquisition process; The specific analysis method of the acquisition deviation degree coefficient is as follows: Step S211: Read the data points of a certain physical environment data collected within a period of time and calculate the average value of a certain physical environment parameter. ,in, Indicates The average value of the physical environment parameters, Indicates The physical environment parameters measurements, Indicates the total number of values ​​of this measurement read over a period of time; Step S212: Based on the average value of a physical environment parameter calculated over a period of time, the standard deviation corresponding to the physical environment parameter is calculated as ; Step S213: Based on the analysis of the average value of a certain physical environment parameter and the standard deviation corresponding to a certain physical environment parameter, the acquisition deviation coefficient is comprehensively calculated as ,in, Indicates The collection deviation coefficient of each physical environment parameter; By taking the collection deviation coefficient Deviation threshold from preset Comparison is performed to determine whether there is a collection error in the data collection of a physical environment parameter. At the same time, it is used to perform parallel error detection on all collected physical environment parameters. If the collection deviation coefficient Preset deviation thresholds , indicating that the data collection of the physical environment parameter is normal, and continue to execute S4 to process the data lightweight. If the collection deviation coefficient Preset deviation thresholds , indicating that there is an error in the data collection of the physical environment parameter, the collection process of the physical environment data is immediately stopped, and all physical environment parameters with collection errors are screened out, and S3 is further executed to detect the data collection abnormal value of the physical environment parameter.

[0007] Preferably, S3 is based on respectively analyzing the abnormal values ​​of the physical environment parameters detected to have acquisition errors, and calculating the deviation between the actual measured value and the expected value of each data point of a physical environment parameter. The specific calculation formula is: ,in, Indicates The physical environment parameters measurements, Indicates The expected value of a physical environment data; the deviation value between the actual measured value and the expected value of each data point of a physical environment parameter , and the coefficient of degree of collection deviation By comparison, all the abnormal values ​​of a certain physical environment parameter collected within a period of time are detected. , then the detected data point is not an outlier, and continue to detect the next data point. If , then the detected data point is an outlier, and all detected outliers are marked as outliers. Then, based on the distribution of the deviation value, the error type generated is determined, including systematic error and random error. At the same time, the outlier detection result and the collection error type are warned, and all detected collection outliers are sent to the management terminal.

[0008] Preferably, the operation process of preprocessing the collected physical environment data in S4 is: removing obviously erroneous, incomplete or irrelevant data, checking the consistency and accuracy of the data, and converting the data into a unified format through linear transformation. The specific conversion formula is: , Represents the original data, Represents the converted data, and Represents the transformation coefficient; the preprocessed physical environment data is subjected to dimensionality reduction processing by the principal component analysis method, and the specific formula is: , represents the covariance matrix of the data, Represents the eigenvector of the covariance matrix, and then uses Huffman coding to compress the reduced-dimensional data.

[0009] Preferably, S5 is used to use a network monitoring tool to collect network data during physical environment data transmission in real time, including data delay, packet loss rate and throughput, and calculate the network congestion control coefficient by analyzing the network data during data transmission. The specific calculation formula is: ,in, Indicates the throughput of data transmission. Indicates the maximum throughput of the network, Indicates the data delay during data transmission. Indicates the maximum acceptable delay of the network. Indicates the packet loss rate during data transmission; By setting the network congestion control coefficient With the preset congestion threshold Compare to determine whether there is network congestion during data transmission. , it indicates that the current data transmission network status is good, continue to monitor the data transmission process, if , it indicates that there is a network congestion problem in the current data transmission, and the network congestion problem is adjusted immediately, including adjusting and reducing the data sending rate, adjusting the size of the congestion window according to the network congestion control coefficient, and replacing the dynamic route. At the same time, the monitoring results of the network congestion are fed back to the management terminal.

[0010] Preferably, S6 analyzes and calculates the storage optimization index by collecting utilization, access logs, costs, and fault records of the current storage system; The specific analysis method of the storage optimization index is as follows: Step S611: Analyze and calculate the storage utilization , Indicates the actual amount of data stored. Indicates the total storage capacity; Step S612: Analyze and calculate the data access efficiency to be , Indicates the number of valid accesses during the current data transmission. Indicates the total number of visits; Step S613: Analyze and calculate the storage cost efficiency to be , represents the storage cost, Indicates the value of data; Step S614: Analyze and calculate the data reliability. , Indicates the data loss rate; Step S615: Based on the above analysis process, the storage optimization index is comprehensively calculated as ,in, represents the weight coefficient; By setting a storage optimization threshold , the storage optimization index Storage Optimization Thresholds Compare to determine whether data storage needs to be optimized. , it indicates that the current data storage status is good and no storage optimization is required. Continue to monitor the storage status. , it indicates that the current data storage status is poor, and the data storage optimization measures are automatically triggered. At the same time, the data storage optimization results are sent to the management terminal for feedback.

[0011] Technical effects and advantages of the present invention: The present invention automatically triggers the physical environment data collection by setting a preset time interval on the data collection device, and sets the delayed response time of the data collection, reads the collected physical environment data from the data collection device, monitors the data flow changes during data collection in real time, and preliminarily analyzes the degree of deviation of the data collection to detect whether there is a collection error in the data collection, detects the abnormal value of the physical environment data through the analysis result of the data collection error, and feeds back the detection result, pre-processes the collected physical environment data, and performs lightweight processing on the pre-processed physical environment data, optimizes and analyzes the data transmission process of the lightweight processed physical environment data, and calculates the network congestion control coefficient, which is used for real-time monitoring and optimization of the data transmission process, based on the data transmission The storage of physical environment data is optimized and analyzed based on the results of data transmission optimization to obtain the storage optimization index, thereby optimizing data storage in real time. By setting the delay threshold for data collection, it is helpful to detect abnormal problems that occur during the data collection process in a timely manner, and can quickly respond to and eliminate data collection failures, thereby achieving rapid and accurate collection of multiple parameters of the physical environment and improving the efficiency of data collection. By real-time monitoring of changes in data flow during data collection, it is possible to promptly discover errors in the data collection process, understand the distribution of abnormal values ​​in the collected data, and automatically determine the type of errors caused by data collection. By automatically optimizing and analyzing data transmission and storage, it is possible to optimize and adjust data transmission and storage based on the analysis results, thereby reducing data maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a diagram of the method steps of the present invention.

[0013] Figure 2 It is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are merely illustrative. The physical environment lightweight data collection method involved in the present invention is not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0015] like Figure 1 This embodiment provides a method for collecting lightweight data of a physical environment, including: S1: Automatically trigger physical environment data collection by setting a preset time interval on the data collection device, and set a delayed response time for data collection.

[0016] In this embodiment, the S1 is used to automatically trigger the collection of physical environment data and detect abnormal delays during data collection. The data collection device selects a multi-channel sensor according to the measurement object to collect physical environment data in parallel, and uses an embedded system to configure a timer based on the collection frequency requirements of the data collection device, sets a specific time interval in the timer, and triggers an interrupt service routine when the timer overflows, thereby performing timed collection of physical environment data, recording the trigger start time and end time of each data collection, and calculating the actual time of each data collection based on the trigger start time and end time of the data collection. The specific calculation formula is: , by taking the actual time of each data collection With delayed response time Comparison is used to detect whether there are abnormal problems in data collection. When the system fails to complete the data collection within the preset delay time, it will immediately enter the warning state and send a warning reminder of data collection delay to the management personnel, thereby eliminating the abnormal problem of data collection. When , it indicates that there is no abnormal problem in data collection caused by delay, and the deviation of data collection is further analyzed.

[0017] S2: By reading the collected physical environment data from the data acquisition device and monitoring the data flow changes during data acquisition in real time, the deviation degree of data acquisition is preliminarily analyzed to obtain the acquisition deviation degree coefficient, which is used to detect whether there is an acquisition error in data acquisition.

[0018] In this embodiment, the S2 reads the data stream from the data acquisition device in real time, records the timestamp of the data acquisition, and then performs preprocessing operations on the read physical environment data. The specific operation process includes data cleaning, data conversion and data smoothing. By using statistical methods to analyze the change trend of the data stream within a period of acquisition time, and calculating the acquisition deviation degree coefficient, the acquisition deviation degree coefficient is used to preliminarily detect whether there is an acquisition error in the data acquisition process; The specific analysis method of the acquisition deviation degree coefficient is as follows: Step S211: Read the data points of a certain physical environment data collected within a period of time and calculate the average value of a certain physical environment parameter. ,in, Indicates The average value of the physical environment parameters, Indicates The physical environment parameters measurements, Indicates the total number of values ​​of this measurement read over a period of time; Step S212: Based on the average value of a physical environment parameter calculated over a period of time, the standard deviation corresponding to the physical environment parameter is calculated as ; Step S213: Based on the analysis of the average value of a certain physical environment parameter and the standard deviation corresponding to a certain physical environment parameter, the acquisition deviation coefficient is comprehensively calculated as ,in, Indicates The collection deviation coefficient of each physical environment parameter; By taking the collection deviation coefficient Deviation threshold from preset Comparison is performed to determine whether there is a collection error in the data collection of a physical environment parameter. At the same time, it is used to perform parallel error detection on all collected physical environment parameters. If the collection deviation coefficient Preset deviation thresholds , indicating that the data collection of the physical environment parameter is normal, and continue to execute S4 to process the data lightweight. If the collection deviation coefficient Preset deviation thresholds , indicating that there is an error in the data collection of the physical environment parameter, the collection process of the physical environment data is immediately stopped, and all physical environment parameters with collection errors are screened out, and S3 is further executed to detect the data collection abnormal value of the physical environment parameter.

[0019] It should be specifically noted that the type of error in data collection can be determined based on the distribution of the deviation values. If the deviation values ​​of the data points are distributed irregularly and evenly, and most of the deviation values ​​are close to CV, this indicates that the error in data collection may be a random error. If the deviation values ​​of the data points are continuously too large or too small, and the direction of the deviation is consistent, this indicates that the error in data collection may be a systematic error, which is usually caused by factors such as improper equipment calibration, sensor failure, or incorrect measurement method.

[0020] S3: Based on the analysis results of data collection errors, detect abnormal values ​​of physical environment data and provide feedback on the detection results.

[0021] In this embodiment, S3 is based on analyzing the abnormal values ​​of the physical environment parameters detected with acquisition errors, and calculating the deviation between the actual measured value and the expected value of each data point of a physical environment parameter. The specific calculation formula is: ,in, Indicates The physical environment parameters measurements, Indicates The expected value of a physical environment data; the deviation value between the actual measured value and the expected value of each data point of a physical environment parameter , and the coefficient of degree of collection deviation By comparison, all the abnormal values ​​of a certain physical environment parameter collected within a period of time are detected. , then the detected data point is not an outlier, and continue to detect the next data point. If , then the detected data point is an outlier, and all detected outliers are marked as outliers. Then, based on the distribution of the deviation value, the error type generated is determined, including systematic error and random error. At the same time, the outlier detection result and the collection error type are warned, and all detected collection outliers are sent to the management terminal.

[0022] S4: used to preprocess the collected physical environment data and perform lightweight processing on the preprocessed physical environment data.

[0023] In this embodiment, the operation process of S4 preprocessing the collected physical environment data is: removing obviously erroneous, incomplete or irrelevant data, checking the consistency and accuracy of the data, and converting the data into a unified format through linear transformation. The specific conversion formula is: , Represents the original data, Represents the converted data, and Represents the transformation coefficient; the preprocessed physical environment data is subjected to dimensionality reduction processing by the principal component analysis method, and the specific formula is: , represents the covariance matrix of the data, Represents the eigenvector of the covariance matrix, and then uses Huffman coding to compress the reduced-dimensional data.

[0024] S5: Based on the optimization analysis of the data transmission process of the lightweight physical environment data, the network congestion control coefficient is calculated for real-time monitoring and optimization of the data transmission process.

[0025] In this embodiment, S5 is used to use a network monitoring tool to collect network data during physical environment data transmission in real time, including data delay, packet loss rate and throughput, and calculate the network congestion control coefficient by analyzing the network data during data transmission. The specific calculation formula is: ,in, Indicates the throughput of data transmission. Indicates the maximum throughput of the network, Indicates the data delay during data transmission. Indicates the maximum acceptable delay of the network. Indicates the packet loss rate during data transmission; By setting the network congestion control coefficient With the preset congestion threshold Compare to determine whether there is network congestion during data transmission. , it indicates that the current data transmission network status is good, continue to monitor the data transmission process, if , it indicates that there is a network congestion problem in the current data transmission, and the network congestion problem is adjusted immediately, including adjusting and reducing the data sending rate, adjusting the size of the congestion window according to the network congestion control coefficient, and replacing the dynamic route. At the same time, the monitoring results of the network congestion are fed back to the management terminal.

[0026] S6: Based on the results of data transmission optimization, the storage of physical environment data is optimized and analyzed to obtain a storage optimization index, thereby optimizing data storage in real time.

[0027] In this embodiment, S6 analyzes and calculates the storage optimization index by collecting utilization, access logs, costs, and fault records of the current storage system; The specific analysis method of the storage optimization index is as follows: Step S611: Analyze and calculate the storage utilization , Indicates the actual amount of data stored. Indicates the total storage capacity; Step S612: Analyze and calculate the data access efficiency to be , Indicates the number of valid accesses during the current data transmission. Indicates the total number of visits; Step S613: Analyze and calculate the storage cost efficiency to be , represents the storage cost, Indicates the value of data; Step S614: Analyze and calculate the data reliability. , Indicates the data loss rate; Step S615: Based on the above analysis process, the storage optimization index is comprehensively calculated as ,in, represents the weight coefficient; By setting a storage optimization threshold , the storage optimization index Storage Optimization Thresholds Compare to determine whether data storage needs to be optimized. , it indicates that the current data storage status is good and no storage optimization is required. Continue to monitor the storage status. , it indicates that the current data storage status is poor, and the data storage optimization measures are automatically triggered. At the same time, the data storage optimization results are sent to the management terminal for feedback.

[0028] like Figure 2 The embodiment shown provides an implementation system corresponding to a physical environment lightweight data collection method, including a data collection trigger module, a data deviation detection module, a data anomaly detection module, a data lightweight processing module, a data transmission optimization module and a data storage optimization module, the data collection trigger module is connected to the data deviation detection module, the data deviation detection module is connected to the data anomaly detection module, the data deviation detection module is connected to the data lightweight processing module, the data lightweight processing module is connected to the data transmission optimization module, and the data transmission optimization module is connected to the data storage optimization module.

[0029] The data acquisition trigger module automatically triggers the physical environment data acquisition by setting a preset time interval on the data acquisition device and setting a delayed response time for data acquisition; The data deviation detection module reads the collected physical environment data from the data acquisition device, monitors the data flow changes during data acquisition in real time, preliminarily analyzes the deviation degree of data acquisition, and obtains the acquisition deviation degree coefficient, which is used to detect whether there is acquisition error in data acquisition; The data anomaly detection module detects abnormal values ​​of the physical environment data based on the analysis results of the data collection errors, and feeds back the detection results; The data lightweight processing module is used to preprocess the collected physical environment data and perform lightweight processing on the preprocessed physical environment data; The data transmission optimization module optimizes and analyzes the data transmission process of the physical environment data after lightweight processing, and calculates the network congestion control coefficient for real-time monitoring and optimization of the data transmission process; The data storage optimization module optimizes and analyzes the storage of physical environment data based on the results of data transmission optimization to obtain a storage optimization index, thereby optimizing data storage in real time.

[0030] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0031] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A lightweight data collection method for a physical environment, characterized in that: include: S1: Automatically triggering physical environment data collection by setting a preset time interval on the data collection device and setting a delayed response time for data collection; S2: By reading the collected physical environment data from the data collection device and monitoring the data flow changes during data collection in real time, the deviation degree of data collection is preliminarily analyzed to obtain the collection deviation degree coefficient, which is used to detect whether there is collection error in data collection; S3: Based on the analysis results of data collection errors, detect abnormal values ​​of physical environment data and provide feedback on the detection results; S4: used to pre-process the collected physical environment data and perform lightweight processing on the pre-processed physical environment data; S5: Optimize and analyze the data transmission process based on the lightweight physical environment data, and calculate the network congestion control coefficient for real-time monitoring and optimization of the data transmission process; S6: Based on the results of data transmission optimization, the storage of physical environment data is optimized and analyzed to obtain a storage optimization index, thereby optimizing data storage in real time.

2. A method for collecting lightweight data of a physical environment according to claim 1, characterized in that: The S1 is used to automatically trigger the collection of physical environment data and detect abnormal delays during data collection. The data collection device selects a multi-channel sensor according to the measurement object to collect physical environment data in parallel, and uses an embedded system to configure a timer based on the collection frequency requirements of the data collection device, sets a specific time interval in the timer, and triggers an interrupt service routine when the timer overflows, thereby performing timed collection of physical environment data, recording the trigger start time and end time of each data collection, and calculating the actual time of each data collection based on the trigger start time and end time of the data collection. , by taking the actual time of each data collection With delayed response time Comparison is used to detect whether there are abnormal problems in data collection. When the system fails to complete the data collection within the preset delay time, it will immediately enter the warning state and send a warning reminder of data collection delay to the management personnel, thereby eliminating the abnormal problem of data collection. When , it indicates that there is no abnormal problem in data collection caused by delay, and the deviation of data collection is further analyzed.

3. A method for collecting lightweight data of a physical environment according to claim 1, characterized in that: The S2 reads the data stream from the data acquisition device in real time, records the timestamp of the data acquisition, and then performs preprocessing operations on the read physical environment data. The specific operation process includes data cleaning, data conversion and data smoothing. By using statistical methods to analyze the change trend of the data stream within a period of acquisition time, and calculating the acquisition deviation degree coefficient, the acquisition deviation degree coefficient is used to preliminarily detect whether there is an acquisition error in the data acquisition process; The specific analysis method of the acquisition deviation degree coefficient is as follows: Step S211: Read the data points of a certain physical environment data collected within a period of time and calculate the average value of a certain physical environment parameter. ,in, Indicates The average value of the physical environment parameters, Indicates The physical environment parameters measurements, Indicates the total number of values ​​of this measurement read over a period of time; Step S212: Based on the average value of a physical environment parameter calculated over a period of time, the standard deviation corresponding to the physical environment parameter is calculated as ; Step S213: Based on the analysis of the average value of a certain physical environment parameter and the standard deviation corresponding to a certain physical environment parameter, the acquisition deviation coefficient is comprehensively calculated as ,in, Indicates The collection deviation coefficient of each physical environment parameter; By taking the collection deviation coefficient Deviation threshold from preset Comparison is performed to determine whether there is a collection error in the data collection of a physical environment parameter. At the same time, it is used to perform parallel error detection on all collected physical environment parameters. If the collection deviation coefficient Preset deviation thresholds , indicating that the data collection of the physical environment parameter is normal, and continue to execute S4 to process the data lightweight. If the collection deviation coefficient Preset deviation thresholds , indicating that there is an error in the data collection of the physical environment parameter, the collection process of the physical environment data is immediately stopped, and all physical environment parameters with collection errors are screened out, and S3 is further executed to detect the data collection abnormal value of the physical environment parameter.

4. A method for collecting lightweight data about a physical environment according to claim 1, characterized in that: The S3 is based on analyzing the abnormal values ​​of the physical environment parameters with acquisition errors detected, and calculating the deviation between the actual measured value and the expected value of each data point of a physical environment parameter. The specific calculation formula is: ,in, Indicates The physical environment parameters measurements, Indicates The expected value of a physical environment data; the deviation value between the actual measured value and the expected value of each data point of a physical environment parameter , and the coefficient of degree of collection deviation By comparison, all the abnormal values ​​of a certain physical environment parameter collected within a period of time are detected. , then the detected data point is not an outlier, and continue to detect the next data point. If , then the detected data point is an outlier, and all detected outliers are marked as outliers. Then, based on the distribution of the deviation value, the error type generated is determined, including systematic error and random error. At the same time, the outlier detection result and the collection error type are warned, and all detected collection outliers are sent to the management terminal.

5. The method for collecting lightweight data of a physical environment according to claim 1, characterized in that: The operation process of S4 preprocessing the collected physical environment data is: removing obviously erroneous, incomplete or irrelevant data, checking the consistency and accuracy of the data, and converting the data into a unified format through linear transformation. The specific conversion formula is: , Represents the original data, Represents the converted data, and Represents the transformation coefficient; the preprocessed physical environment data is subjected to dimensionality reduction processing by the principal component analysis method, and the specific formula is: , represents the covariance matrix of the data, Represents the eigenvector of the covariance matrix, and then uses Huffman coding to compress the reduced-dimensional data.

6. A method for collecting lightweight data about a physical environment according to claim 1, characterized in that: The S5 is used to use a network monitoring tool to collect network data during data transmission in the physical environment in real time, including data delay, packet loss rate, and throughput, and calculate the network congestion control coefficient by analyzing the network data during data transmission. The specific calculation formula is: ,in, Indicates the throughput of data transmission. Indicates the maximum throughput of the network, Indicates the data delay during data transmission. Indicates the maximum acceptable delay of the network. Indicates the packet loss rate during data transmission; By setting the network congestion control coefficient With the preset congestion threshold Compare to determine whether there is network congestion during data transmission. , it indicates that the current data transmission network status is good, continue to monitor the data transmission process, if , it indicates that there is a network congestion problem in the current data transmission, and the network congestion problem is adjusted immediately, including adjusting and reducing the data sending rate, adjusting the size of the congestion window according to the network congestion control coefficient, and replacing the dynamic route. At the same time, the monitoring results of the network congestion are fed back to the management terminal.

7. A method for collecting lightweight data about a physical environment according to claim 1, characterized in that: The S6 analyzes and calculates the storage optimization index by collecting utilization, access logs, costs, and fault records of the current storage system; The specific analysis method of the storage optimization index is as follows: Step S611: Analyze and calculate the storage utilization , Indicates the actual amount of data stored. Indicates the total storage capacity; Step S612: Analyze and calculate the data access efficiency to be , Indicates the number of valid accesses during the current data transmission. Indicates the total number of visits; Step S613: Analyze and calculate the storage cost efficiency to be , represents the storage cost, Indicates the value of data; Step S614: Analyze and calculate the data reliability. , Indicates the data loss rate; Step S615: Based on the above analysis process, the storage optimization index is comprehensively calculated as ,in, represents the weight coefficient; By setting a storage optimization threshold , the storage optimization index Storage Optimization Thresholds Compare to determine whether data storage needs to be optimized. , it indicates that the current data storage status is good and no storage optimization is required. Continue to monitor the storage status. , it indicates that the current data storage status is poor, and the data storage optimization measures are automatically triggered. At the same time, the data storage optimization results are sent to the management terminal for feedback.

Images