Intelligent building multi-level collaborative internet-of-things sensing big data analysis method and system

Abstract

The invention relates to the technical field of big data, and discloses a smart building multi-level collaborative internet-of-things sensing big data analysis method and system, and the method comprises the steps: collecting internet-of-things sensing data through a sensor, and carrying out the preprocessing of the internet-of-things sensing data, and obtaining initial sensing data; clustering the initial sensing data through an edge server to obtain multi-source data; determining a correlation coefficient corresponding to any two random variables in the multi-source data through a cloud server, and generating a cumulative distribution function and a probability density function corresponding to any two random variables based on the correlation coefficient; and based on the cumulative distribution function and the probability density function, generating a joint probability function corresponding to any two random variables to perform correlation analysis to obtain a target analysis result. The beneficial effects of the invention are that different user demands are satisfied, assistance is provided for users to make decisions, and the effect of urban fine management is realized.

Description

Technical Field

[0001] The present application relates to the technical field of big data analysis, and in particular to a method and system for multi-level collaborative IoT perception big data analysis in smart buildings. Background Art

[0002] Under the policy background of the "dual carbon" goal, the construction of a zero-carbon building integrated energy system has become an important means to reduce building carbon emissions and help cities achieve "carbon neutrality". With the continuous growth of energy demand for building integrated energy systems, the equipment is flexible and diverse and closely connected. From the perspective of sustainable urban development, it is necessary to explore the potential for energy conservation and emission reduction from the aspects of building source, network, load, storage and resource recycling.

[0003] However, the current zero-carbon building integrated energy system faces multiple challenges such as data silos, complex coupling of heterogeneous energy flows, insufficient source-load prediction accuracy, difficulty in technical coordination, low proportion of renewable energy consumption, and bottlenecks in data cleaning and analysis. It is urgent to effectively analyze the multi-level collaborative IoT perception big data of smart buildings to achieve system optimization and zero-carbon goals. Summary of the invention

[0004] The present application provides a method and system for analyzing big data of multi-level collaborative Internet of Things perception in smart buildings, which solves the technical problem of how to improve the analysis technology of big data of multi-level collaborative Internet of Things perception in smart buildings and prevent data jitter, thereby achieving the technical effect of meeting the needs of different users, providing assistance for users to make decisions, and realizing refined urban management.

[0005] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0006] In a first aspect, an embodiment of the present application provides a method for analyzing big data of multi-level collaborative IoT perception of a smart building, the method comprising:

[0007] Collecting IoT perception data through sensors, and preprocessing the IoT perception data to obtain initial perception data;

[0008] Clustering the initial perception data through an edge server to obtain multi-source data;

[0009] Determine the correlation coefficient corresponding to any two random variables in the multi-source data through the cloud server, and generate the cumulative distribution function and probability density function corresponding to the any two random variables based on the correlation coefficient;

[0010] Based on the cumulative distribution function and the probability density function, a joint probability function corresponding to the arbitrary two random variables is generated to perform correlation analysis to obtain a target analysis result.

[0011] The present embodiment provides a method for analyzing big data of multi-level collaborative IoT perception in a smart building. The IoT perception data is collected through sensors and preprocessed to ensure that the initial perception data obtained is of high quality and high accuracy. Next, the edge server clusters the initial perception data and can classify the data by features to extract meaningful information. Subsequently, the correlation coefficient of any two random variables is calculated on the cloud server to reveal the potential relationship between the variables and support further decision-making. The generated cumulative distribution function and probability density function provide a mathematical basis for understanding data distribution, while the generation of the joint probability function deeply analyzes the relationship between variables and identifies potential dependencies. The overall process significantly improves the accuracy and efficiency of IoT data analysis and provides a solid basis for accurate prediction and decision-making. The embodiment of the present application can realize data collection in multi-type and fully covered "end" sensors, realize multi-source data fusion using the "edge" end, and mine coupling correlation and multi-energy complementarity and other features on the "cloud" end. In order to address the problems of computing power differences and complex collaboration in the "cloud-edge-end" IoT perception system, data fusion at different levels is achieved through the coupling correlation and complementarity of multi-dimensional heterogeneous building intelligent big data, so as to reduce problems such as data loss and data jitter in the "edge-end" process.

[0012] Optionally, the preprocessing of the IoT perception data to obtain initial perception data includes:

[0013] Using an isolation forest algorithm to remove outliers from the IoT perception data to obtain the initial perception data; and / or

[0014] The Internet of Things perception data is enhanced by using a Lagrange interpolation method to obtain the initial perception data.

[0015] Optionally, the using of an isolation forest algorithm to remove outliers from the IoT perception data to obtain the initial perception data includes:

[0016] Sorting the IoT perception data according to a time series, and determining the start and end time of each data source in the IoT perception data;

[0017] Inputting the IoT sensing data and the start and end times into a preset isolation forest model to obtain an anomaly score for each data point in the data source;

[0018] Determine the data point corresponding to the anomaly score being greater than the preset anomaly threshold as an outlier;

[0019] The abnormal value is removed from the IoT perception data to obtain the initial perception data.

[0020] This embodiment can clearly understand the changing trend of data and provide an accurate basis for anomaly detection by implementing the time series sorting and start and end time determination of IoT perception data. Then, the anomaly score of each data point is calculated using the isolation forest model to effectively identify abnormal patterns and improve detection accuracy. By setting an abnormal threshold, data points whose anomaly scores exceed the threshold are marked as outliers, reducing the risk of misjudgment caused by abnormal data. Finally, after removing these outliers, the initial perception data obtained is more reliable, thereby improving the efficiency and accuracy of subsequent data analysis and decision-making.

[0021] Optionally, the interpolating the IoT perception data using a Lagrange interpolation method to obtain the initial perception data includes:

[0022] For the missing values ​​in the IoT sensing data, an estimated value is obtained by using the Lagrange interpolation method;

[0023] The estimated value is inserted into the IoT perception data as a new data point to obtain the initial perception data.

[0024] This embodiment can effectively fill in the data gaps by estimating the missing values ​​in the IoT perception data using the Lagrange interpolation method. This method uses the relationship between known data points to provide accurate estimates, thereby reducing the problem of data incompleteness caused by missing values. Next, the estimated values ​​are inserted into the IoT perception data as new data points, and the generated initial perception data is more complete and continuous. This process not only improves the availability of data, but also provides a more accurate and comprehensive data basis for subsequent analysis and decision-making, thereby improving the quality and efficiency of overall data processing.

[0025] Optionally, the Lagrange interpolation polynomial in the Lagrange interpolation method is:

[0026]

[0027] Among them, x i is the horizontal coordinate of data point i in the IoT perception data; x j is the horizontal coordinate of data point j in the IoT perception data; y j is the ordinate of data point j in the IoT perception data; n is the number of data points in the IoT perception data.

[0028] Optionally, clustering the initial perception data through an edge server to obtain multi-source data includes:

[0029] Performing standardization processing on the initial perception data to obtain standard sample data;

[0030] Determine K samples as the initial clustering centers;

[0031] Clustering processing step: based on a preset distance function, determining the distance between the standard sample data and each of the initial cluster centers, and assigning each standard sample data to the class where the initial cluster center closest to the distance is located;

[0032] Determine the mean of the standard sample data in the class as a new cluster center;

[0033] Compare the error between the new cluster center and the initial cluster center, and if the error does not meet the preset condition, iterate the clustering processing step until the error meets the preset condition, so as to obtain the clustered multi-source data.

[0034] This embodiment first standardizes the initial perception data to eliminate the influence of different dimensions and scales on the clustering results, thereby obtaining standard sample data. Then, K samples are randomly selected as the initial clustering centers. Through the preset distance function, the standard sample data is assigned to the clustering center closest to it to form a preliminary cluster. Subsequently, by calculating the mean of the samples in each class, the clustering center is updated to better represent the characteristics of each class. Finally, the error between the new and old clustering centers is compared, and when the error does not meet the preset conditions, the clustering process is iterated until convergence, and finally an accurate and reliable clustering result is obtained, thereby effectively improving the clustering effect of multi-source data.

[0035] Optionally, determining the correlation coefficient corresponding to any two random variables in the multi-source data through the cloud server includes:

[0036] Determine the Kendall rank correlation coefficient corresponding to any two random variables in the multi-source data;

[0037] The correlation coefficient corresponding to the arbitrary two random variables is generated according to the Kendall rank correlation coefficient.

[0038] This embodiment determines the Kendall rank correlation coefficient of any two random variables in multi-source data, which can effectively evaluate the correlation between variables, especially when processing non-parametric data, with high robustness and accuracy. This enables analysts to identify the degree of association between variables and reveal potential patterns or relationships. Subsequently, the correlation coefficient is generated based on the Kendall rank correlation coefficient, which further quantifies the correlation between variables and provides a clear numerical basis for subsequent data analysis and decision-making. In this way, the interaction analysis between variables in complex systems can be effectively supported, providing an empirical basis for optimizing decision-making.

[0039] Optionally, the relationship between the correlation coefficient and the Kendall rank correlation coefficient is:

[0040]

[0041] Among them, x is the independent variable, τ is the Kendall rank correlation coefficient, and θ is the correlation coefficient.

[0042] Optionally, the cumulative distribution function is:

[0043]

[0044] The probability density function is:

[0045]

[0046] Among them, x is the independent variable, x∈[0,1]; θ is the correlation coefficient, θ≠0, θ<0 means that the random variables u and v are positively correlated, θ>0 means that the random variables u and v are negatively correlated, and the closer θ is to 0, the more independent the random variables u and v tend to be.

[0047] In a second aspect, an embodiment of the present application provides a smart building multi-level collaborative IoT perception big data analysis system, the system comprising:

[0048] A data acquisition unit, used to collect IoT perception data through sensors, and pre-process the IoT perception data to obtain initial perception data;

[0049] A clustering processing unit, configured to cluster the initial perception data through an edge server to obtain multi-source data;

[0050] A function generating unit, configured to determine, through a cloud server, a correlation coefficient corresponding to any two random variables in the multi-source data, and generate a cumulative distribution function and a probability density function corresponding to the any two random variables based on the correlation coefficient;

[0051] The correlation analysis unit generates a joint probability function corresponding to any two random variables based on the cumulative distribution function and the probability density function to perform correlation analysis and obtain a target analysis result. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1A flowchart of a smart building multi-level collaborative IoT perception big data analysis method provided in an embodiment of the present application;

[0054] Figure 2 A flowchart of an isolation forest algorithm provided in an embodiment of the present application;

[0055] Figure 3 A flowchart using the Lagrange interpolation method provided in an embodiment of the present application;

[0056] Figure 4 A flowchart of step S3 provided in an embodiment of the present application;

[0057] Figure 5 A flowchart of determining the correlation coefficient corresponding to any two random variables in multi-source data through a cloud server provided in an embodiment of the present application;

[0058] Figure 6 This is a comparison chart of heat load sea data before and after preprocessing;

[0059] Figure 7 This is a comparison chart of the electric load sea data before and after preprocessing;

[0060] Figure 8 is the joint probability distribution of air temperature and heat load;

[0061] Fig. 9 is the joint probability distribution of human and vehicle flow;

[0062] Fig.10 It is the joint probability distribution of traffic and electricity;

[0063] Fig.11 A smart building multi-level collaborative Internet of Things perception big data analysis system is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0065] Under the policy background of the "dual carbon" goal, the construction of a zero-carbon building integrated energy system has become an important means to reduce building carbon emissions and help cities achieve "carbon neutrality". With the continuous growth of energy demand for building integrated energy systems, the equipment is flexible and diverse and closely connected. From the perspective of sustainable urban development, it is necessary to explore the potential for energy conservation and emission reduction from the aspects of building source, network, load, storage and resource recycling.

[0066] Multi-dimensional data analysis of smart building energy supply is the basis for exploring multi-energy coupling, reducing energy consumption and carbon reduction in urban buildings. In view of the high coupling and response time differences of multiple heterogeneous energy flows such as cold, heat, electricity and gas in the integrated energy system of urban buildings, a "cloud-edge-end" multi-level collaborative multi-dimensional energy data model is constructed through data analysis of heterogeneous sources-loads-people-vehicles-weather. By using data fusion and big data technology, accurate source-load prediction can be achieved, the distributed resource allocation of zero-carbon buildings can be optimized, and the economy of the integrated energy system can be improved. IoT sensing networks, data fusion and feature mining technologies can significantly improve the source-load prediction accuracy of integrated energy systems, increase the proportion of renewable energy consumption, and provide support for system optimization. By cleaning and extracting multi-format cloud-end data, analyzing invalid and missing data, and completing the clustering and correlation calculation of heterogeneous sources and loads, the foundation is laid for the low-carbon and high-efficiency operation analysis of buildings.

[0067] In addition, a "cloud-edge-end" Internet of Things perception system is built to detect data through multiple types of sensors, realize multi-source data fusion on the "edge" end, and explore coupling correlation and multi-energy complementarity on the "cloud" end. In order to address problems such as differences in computing power and complex collaboration, the stability of data fusion is ensured and data loss and jitter are reduced.

[0068] According to an embodiment of the present application, an embodiment of a smart building multi-level collaborative Internet of Things perception big data analysis method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0069] In this embodiment, a method for analyzing big data of multi-level collaborative IoT perception of smart buildings is provided. Figure 1 A flowchart of a smart building multi-level collaborative IoT perception big data analysis method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the process includes the following steps:

[0070] Step S1, collect IoT perception data through sensors, and pre-process the IoT perception data to obtain initial perception data.

[0071] Specifically, in the multi-level collaborative IoT perception big data analysis method for smart buildings, IoT perception data is collected through sensors. These IoT perception data include data on sources, loads, people, vehicles, and the environment. Among them, source refers to the energy supply side, including the supply of various energy sources such as electricity, heat, and natural gas. Specific data include power generation, power supply quality, peak and valley periods of power consumption, temperature and pressure of heat supply, and natural gas flow. Load refers to the energy demand side, involving energy consumption data of equipment and systems that consume energy inside the building, such as air conditioning, lighting, and elevators. These data include energy consumption, usage frequency, operating status, and efficiency of the equipment. People refer to personnel behavior patterns, involving the behavior patterns of personnel in the building, such as in and out flow, distribution of activity areas, and habits of using facilities. Vehicles refer to vehicle behavior patterns, involving the flow of vehicles inside or around the building, including vehicle entry and exit records, parking conditions, vehicle types, and flow. Environment refers to environmental parameters, including indoor and outdoor environmental parameters, such as temperature, humidity, air quality, light intensity, and noise level.

[0072] In some preferred embodiments, an isolation forest algorithm is used to remove outliers from IoT perception data to obtain initial perception data; and / or

[0073] The Lagrange interpolation method is used to enhance the IoT perception data and obtain the initial perception data.

[0074] Specifically, since the collected IoT perception data may have abnormal values ​​or missing data due to problems with sensor devices, it is necessary to preprocess these data. The preprocessing content includes but is not limited to outlier removal, data supplementation and data enhancement, so as to obtain the initial perception data after preprocessing.

[0075] Step S3: clustering the initial perception data through the edge server to obtain multi-source data.

[0076] Specifically, the initial perception data obtained after preprocessing is uploaded from the end sensor to the edge server at the edge. The edge server needs to analyze the impact of different energy unit attributes, weather, working days and non-working days on building energy consumption, and consider multi-time scale characteristics (such as source, load, traffic, flow of people and weather). In other words, it is not limited to short-term data, but covers changes in different time periods for analysis. Therefore, it is necessary to cluster the initial perception data, and the clustering method can use the K-means clustering algorithm.

[0077] Step S5, determining the correlation coefficient corresponding to any two random variables in the multi-source data through the cloud server, and generating the cumulative distribution function and probability density function corresponding to any two random variables based on the correlation coefficient.

[0078] Specifically, the multi-source data obtained after clustering is uploaded to the cloud server, and the cloud server analyzes the source-load and its coupling characteristics with pedestrian flow, vehicle flow, environment, etc. Here, the multi-source data can be analyzed through the cloud server to calculate the correlation coefficient between any two random variables. After obtaining the correlation coefficient, this coefficient can be used to generate the cumulative distribution function and probability density function of the two random variables, which are calculated and generated using the Frank Copula function.

[0079] Step S7, based on the cumulative distribution function and the probability density function, generate a joint probability function corresponding to any two random variables to perform correlation analysis and obtain a target analysis result.

[0080] Specifically, through the cumulative distribution function and the probability density function, a joint probability distribution diagram can be drawn, such as a contour diagram or a three-dimensional surface diagram, to intuitively display the joint probability distribution of two random variables, which helps to further analyze and understand the dependency structure and correlation between the two variables and obtain the target analysis result, which is a visualization chart.

[0081] In some preferred embodiments, the cumulative distribution function is:

[0082]

[0083] The probability density function is:

[0084]

[0085] Among them, x is the independent variable, x∈[0,1]; θ is the correlation coefficient, θ≠0, θ<0 means that the random variables u and v are positively correlated, θ>0 means that the random variables u and v are negatively correlated, and the closer θ is to 0, the more independent the random variables u and v tend to be.

[0086] The generators of the Frank Copula function are:

[0087]

[0088] Specifically, the Frank Copula function is a statistical tool used to describe the dependence between multiple random variables. To describe the dependency structure between random variables and to construct the Copula function. The dimensions of multi-source data are different, so they need to be standardized so that their values ​​are within the interval [0,1]. Finally, the target analysis results obtained through the above formula can reveal the potential relationship between different factors and provide data support for subsequent decision-making. For example, in building energy consumption management, traffic flow prediction and environmental monitoring, this analysis can help optimize resource allocation and improve efficiency.

[0089] The present embodiment provides a method for analyzing big data of multi-level collaborative IoT perception in a smart building. The IoT perception data is collected through sensors and preprocessed to ensure that the initial perception data obtained is of high quality and high accuracy. Next, the edge server clusters the initial perception data and can classify the data by features to extract meaningful information. Subsequently, the correlation coefficient of any two random variables is calculated on the cloud server to reveal the potential relationship between the variables and support further decision-making. The generated cumulative distribution function and probability density function provide a mathematical basis for understanding data distribution, while the generation of the joint probability function deeply analyzes the relationship between variables and identifies potential dependencies. The overall process significantly improves the accuracy and efficiency of IoT data analysis and provides a solid basis for accurate prediction and decision-making. The embodiment of the present application can realize data collection in multi-type and fully covered "end" sensors, realize multi-source data fusion using the "edge" end, and mine coupling correlation and multi-energy complementarity and other features on the "cloud" end. In order to address the problems of computing power differences and complex collaboration in the "cloud-edge-end" IoT perception system, data fusion at different levels is achieved through the coupling correlation and complementarity of multi-dimensional heterogeneous building intelligent big data, so as to reduce problems such as data loss and data jitter in the "edge-end" process.

[0090] Figure 2 The flowchart of using the isolation forest algorithm provided in the embodiment of the present application may include the following steps:

[0091] Step S111, sorting the IoT perception data according to the time series, and determining the start and end time of each data source in the IoT perception data.

[0092] Step S113, input the IoT sensing data and the start and end time into a preset isolation forest model to obtain an anomaly score for each data point in the data source.

[0093] Step S115: determine the data points corresponding to the anomaly scores greater than the preset anomaly threshold as outliers.

[0094] Step S117, remove the abnormal values ​​from the IoT perception data to obtain initial perception data.

[0095] The Isolation Forest Algorithm defines outliers as those that are easily isolated in the data space, that is, those that are sparsely distributed and far away from high-density groups. Usually, the proportion of abnormal data in IoT perception data is very small, and the characteristic values ​​of abnormal points are very different from those of normal points.

[0096] Specifically, first, the IoT perception data is sorted according to the time series, and the start and end time of each data source is determined. Then, the sorted time series data and the start and end time are input into the preset isolation forest model, and the model calculates the anomaly score for each data point. An anomaly threshold is set to mark the data points with anomaly scores greater than the threshold as outliers. Subsequently, these outliers are removed from the IoT perception data, and the remaining data is regarded as clean initial perception data. It should be noted that, considering the uniqueness of each data source, including their distribution, range of variation, and abnormal pattern, an independent anomaly threshold can be set for each data source to more accurately identify outliers. The anomaly threshold is usually set through expert knowledge.

[0097] and Figure 1 Compared with the embodiment shown in the figure, this embodiment can clearly understand the changing trend of the data by implementing the time series sorting and start and end time determination of the IoT perception data, providing an accurate basis for anomaly detection. Then, the anomaly score of each data point is calculated using the isolation forest model to effectively identify abnormal patterns and improve detection accuracy. By setting an abnormal threshold, data points whose anomaly scores exceed the threshold are marked as outliers, reducing the risk of misjudgment caused by abnormal data. Finally, after removing these outliers, the initial perception data obtained is more reliable, thereby improving the efficiency and accuracy of subsequent data analysis and decision-making.

[0098] Figure 3 The flowchart using the Lagrange interpolation method provided in the embodiment of the present application may include the following steps:

[0099] Step S131, for the missing values ​​in the IoT sensing data, an estimated value is obtained using the Lagrange interpolation method.

[0100] In some preferred embodiments, the Lagrange interpolation polynomial in the Lagrange interpolation method is:

[0101]

[0102] Among them, x i is the horizontal coordinate of data point i in the IoT perception data; x j is the horizontal coordinate of data point j in the IoT perception data; y j is the ordinate of data point j in the IoT perception data; n is the number of data points in the IoT perception data.

[0103] Step S132: insert the estimated value as a new data point into the IoT perception data to obtain initial perception data.

[0104] Specifically, we determine the known data points in the IoT sensing data, which are represented by (x1, y1), (x2, y2),…, (xn ,y n ). For each known data point (x i ,y i ), construct the corresponding Lagrange interpolation polynomial L i (x), when x=x i The value is 1 when x j The value is 0.

[0105]

[0106] Next, using the y of the known data points i The values ​​are used as coefficients to linearly combine all the basis polynomials to form the Lagrange interpolation polynomial This polynomial is at every x i The value at is equal to y i , that is, L(x i )=y i For the missing value x of missing data, substitute x into L(x) and calculate the corresponding y value, which is the estimated value of the missing data.

[0107] Finally, the estimated values ​​obtained by Lagrange interpolation are inserted into the IoT perception data as new data points to fill in the missing values. After filling in the missing values, a complete initial perception data set is obtained.

[0108] and Figure 1 Compared with the embodiment shown in the figure, this embodiment can effectively fill the data gaps by estimating the missing values ​​in the IoT perception data using the Lagrange interpolation method. This method uses the relationship between known data points to provide accurate estimates, thereby reducing the problem of data incompleteness caused by missing values. Next, the estimated values ​​are inserted into the IoT perception data as new data points, and the generated initial perception data is more complete and continuous. This process not only improves the availability of data, but also provides a more accurate and comprehensive data basis for subsequent analysis and decision-making, thereby improving the quality and efficiency of overall data processing.

[0109] Figure 4 The flowchart of step S3 provided in the embodiment of the present application may include the following steps:

[0110] Step S31, standardize the initial perception data to obtain standard sample data.

[0111] Specifically, the initial perception data is standardized to eliminate the dimensional effects between different features so that the data can be compared at the same scale. The standardization process can be Min-Max standardization, Z-score standardization, maximum absolute value standardization, etc.

[0112] In some preferred embodiments, the normalization process uses Z-score normalization.

[0113] Specifically, Z-score standardization is used, that is, each data point is subtracted from its mean and divided by its standard deviation, so that the mean of the data is 0 and the standard deviation is 1.

[0114] Step S33, determining K samples as initial cluster centers.

[0115] Step S35, clustering processing step: based on a preset distance function, determine the distance between the standard sample data and each initial cluster center, and assign each standard sample data to the class where the initial cluster center with the closest distance is located.

[0116] Step S37, determining the mean of the standard sample data in the class as the new cluster center.

[0117] Step S39, comparing the error between the new cluster center and the initial cluster center, and when the error does not meet the preset condition, iterating the clustering processing step until the error meets the preset condition, thereby obtaining the clustered multi-source data.

[0118] Specifically, determine the number of clusters K, that is, you want to divide the data into K categories. Randomly select K samples from the standardized data as initial cluster centers. According to a preset distance function, usually the Euclidean distance, calculate the distance between each standard sample data and each initial cluster center. Assign each sample to the class where the cluster center closest to it is located. For each class, calculate the mean of all standard sample data assigned to the class, and determine this mean as the new cluster center. Compare the error between the new cluster center and the initial cluster center. If the error does not meet the preset conditions, return to step S35 to continue iterating. The preset conditions can set an error threshold or a maximum number of iterations. If the error meets the preset conditions, that is, the cluster center no longer changes or changes very little, the clustering process ends.

[0119] and Figure 1 Compared with the embodiment shown in the figure, this embodiment first standardizes the initial perception data to eliminate the influence of different dimensions and scales on the clustering results, thereby obtaining standard sample data. Then, K samples are randomly selected as the initial clustering centers. Through the preset distance function, the standard sample data is assigned to the clustering center closest to it to form a preliminary cluster. Subsequently, by calculating the mean of the samples in each class, the clustering center is updated to better represent the characteristics of each class. Finally, the error between the new and old clustering centers is compared, and when the error does not meet the preset conditions, the clustering process is iterated until convergence, and finally an accurate and reliable clustering result is obtained, thereby effectively improving the clustering effect of multi-source data.

[0120] Figure 5 A flowchart of determining the correlation coefficient corresponding to any two random variables in multi-source data through a cloud server provided in an embodiment of the present application may include the following steps:

[0121] Step S51, determining the Kendall rank correlation coefficient corresponding to any two random variables in the multi-source data.

[0122] Specifically, the data of any two variables are ranked separately. If there are identical values ​​in the data, i.e. ties, the average rank is assigned. Determine concordant pairs: pairs where the ranking order of the two variables is the same. For example, if one data point is ranked higher than another data point in variable X, and is also ranked higher in variable Y, then the two data points form a concordant pair. Divergent pairs: pairs where the ranking order of the two variables is opposite. For example, if one data point is ranked higher than another data point in variable X, but is ranked lower in variable Y, then the two data points form a divergent pair.

[0123] The Kendall rank correlation coefficient is calculated using the following formula:

[0124]

[0125] Where C is the number of consistent pairs, D is the number of divergent pairs, and T t is the number of tied rankings in variable X, T f is the number of ties in variable Y.

[0126] The Kendall rank correlation coefficient ranges from -1 to +1. Values ​​close to 1 indicate a strong positive correlation, values ​​close to -1 indicate a strong negative correlation, and values ​​close to 0 indicate no correlation.

[0127] Step S53: Generate a correlation coefficient corresponding to any two random variables according to the Kendall rank correlation coefficient.

[0128] In some preferred embodiments, the relationship between the correlation coefficient and the Kendall rank correlation coefficient is:

[0129]

[0130] Where τ is the Kendall rank correlation coefficient.

[0131] Specifically, D(θ) is used to adjust the value of θ to match the actual value of the Kendall rank correlation coefficient τ, thereby analyzing the correlation between the two random variables. The correlation coefficient θ can be inferred from the above formula.

[0132] and Figure 1Compared with the embodiment shown, the present embodiment determines the Kendall rank correlation coefficient of any two random variables in multi-source data, which can effectively evaluate the correlation between variables, especially when processing non-parametric data, with high robustness and accuracy. This enables analysts to identify the degree of association between variables and reveal potential patterns or relationships. Subsequently, the correlation coefficient is generated based on the Kendall rank correlation coefficient, which further quantifies the correlation between variables and provides a clear numerical basis for subsequent data analysis and decision-making. In this way, the interaction analysis between variables in complex systems can be effectively supported, providing an empirical basis for optimizing decision-making.

[0133] The following describes the specific implementation of the present invention in conjunction with a specific application scenario. In this specific application scenario, taking the "cloud-edge-end" multi-level and multi-dimensional intelligent data of the source-load-people-vehicle-environment of a company building as an example, a correlation analysis is performed on renewable energy power generation data, load data, and human and vehicle flow data in combination with weather information. Figure 6-Figure 7 The figure is a comparison chart before and after data preprocessing; the rank correlation coefficients of Kendall and Spearman rank correlation coefficients calculated by fitting the Frank Copula function are shown in Table 1:

[0134] Table 1 Rank correlation coefficient of Copula function

[0135] Copula Function Spearman rank Kendall Rank Frank Copula -0.1550 -0.1036 Raw data -0.1226 -0.0480

[0136] By drawing the joint distribution function diagram and joint density function diagram of heat supply and temperature, the correlation measure of the multi-level heterogeneous cloud Frank Copula function proposed in the embodiment of the present application is verified. After generating the Copula function model, the Kendall and Spearman rank correlation coefficients are calculated respectively to test the goodness of fit of the Frank Copula function. The closer the rank correlation coefficient is to the rank correlation coefficient of the original data, the better the goodness of fit of the Copula function.

[0137] When conducting correlation analysis on renewable energy generation data, load data, and pedestrian and vehicle flow data collected by the building's intelligent system, the heat and temperature data are normalized. Figure 8 , Fig. 9 and Fig.10 The probability density distributions of air conditioning heating and pedestrian flow, vehicle flow and pedestrian flow, and electricity consumption and pedestrian flow are shown respectively. The analysis results show that there is a strong correlation and symmetry between temperature and heating supply; the goodness of fit between pedestrian flow and vehicle flow is good and the positive correlation is significant, which is in line with the actual situation and has weak randomness, indicating that during periods of high pedestrian flow, vehicle traffic also increases. In addition, there is also a strong positive correlation between electricity consumption and pedestrian flow.

[0138] Accordingly, please refer to Fig.11 A block diagram of a smart building multi-level collaborative IoT perception big data analysis system provided in an embodiment of the present application, the system includes:

[0139] The data collection unit S101 is used to collect IoT perception data through sensors and pre-process the IoT perception data to obtain initial perception data;

[0140] A clustering processing unit S103 is used to cluster the initial perception data through the edge server to obtain multi-source data;

[0141] The function generating unit S105 is used to determine the correlation coefficient corresponding to any two random variables in the multi-source data through the cloud server, and generate the cumulative distribution function and probability density function corresponding to any two random variables based on the correlation coefficient;

[0142] The correlation analysis unit S107 generates a joint probability function corresponding to any two random variables based on the cumulative distribution function and the probability density function to perform correlation analysis and obtain a target analysis result.

[0143] In some optional implementations, the data acquisition unit S101 includes:

[0144] Using the isolation forest algorithm to remove outliers from IoT perception data to obtain initial perception data; and / or

[0145] The Lagrange interpolation method is used to enhance the IoT perception data and obtain the initial perception data.

[0146] In some optional implementations, an isolation forest algorithm is used to remove outliers from IoT perception data to obtain initial perception data, including:

[0147] Sort the IoT perception data in time series and determine the start and end time of each data source in the IoT perception data;

[0148] Input IoT sensing data and start and end times into the preset isolation forest model to obtain the anomaly score of each data point in the data source;

[0149] The data points corresponding to the anomaly scores greater than the preset anomaly threshold are determined as outliers;

[0150] The outliers are removed from the IoT perception data to obtain the initial perception data.

[0151] In some optional implementations, the Internet of Things perception data is interpolated using the Lagrange interpolation method to obtain initial perception data, including:

[0152] For the missing values ​​in IoT sensing data, the Lagrange interpolation method is used to obtain the estimated values;

[0153] The estimated value is inserted into the IoT perception data as a new data point to obtain the initial perception data.

[0154] In some optional implementations, the Lagrange interpolation polynomial in the Lagrange interpolation method is:

[0155]

[0156] Among them, x i is the horizontal coordinate of data point i in the IoT perception data; x j is the horizontal coordinate of data point j in the IoT perception data; y j is the ordinate of data point j in the IoT perception data; n is the number of data points in the IoT perception data.

[0157] In some optional implementations, the clustering processing unit S103 includes:

[0158] Standardize the initial perception data to obtain standard sample data;

[0159] Determine K samples as the initial clustering centers;

[0160] Clustering processing step: based on a preset distance function, determine the distance between the standard sample data and each initial cluster center, and assign each standard sample data to the class where the initial cluster center with the closest distance is located;

[0161] Determine the mean of the standard sample data in the class as the new cluster center;

[0162] The error between the new cluster center and the initial cluster center is compared. If the error does not meet the preset condition, the clustering processing steps are iterated until the error meets the preset condition to obtain the clustered multi-source data.

[0163] In some optional implementations, determining the correlation coefficient corresponding to any two random variables in the multi-source data by the cloud server includes:

[0164] Determine the Kendall rank correlation coefficient between any two random variables in multi-source data;

[0165] Based on the Kendall rank correlation coefficient, generate the correlation coefficient corresponding to any two random variables.

[0166] In some optional embodiments, the relationship between the correlation coefficient and the Kendall rank correlation coefficient is:

[0167]

[0168] Among them, x is the independent variable, τ is the Kendall rank correlation coefficient, and θ is the correlation coefficient.

[0169] In some optional embodiments, the cumulative distribution function is:

[0170]

[0171] The probability density function is:

[0172]

[0173] Among them, x is the independent variable, x∈[0,1]; θ is the correlation coefficient, θ≠0, θ<0 means that the random variables u and v are positively correlated, θ>0 means that the random variables u and v are negatively correlated, and the closer θ is to 0, the more independent the random variables u and v tend to be.

[0174] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0175] In this embodiment, a smart building multi-level collaborative IoT perception big data analysis system is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0176] The systems or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0177] For the convenience of description, the above system is described by dividing it into various units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0178] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods and systems. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] The present application is described with reference to the flowcharts and / or block diagrams of the methods and systems according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0182] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0184] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0185] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for analyzing big data of multi-level collaborative IoT perception in smart buildings, characterized in that: The method comprises: Collecting IoT perception data through sensors, and preprocessing the IoT perception data to obtain initial perception data; Clustering the initial perception data through an edge server to obtain multi-source data; Determine the correlation coefficient corresponding to any two random variables in the multi-source data through the cloud server, and generate the cumulative distribution function and probability density function corresponding to the any two random variables based on the correlation coefficient; Based on the cumulative distribution function and the probability density function, a joint probability function corresponding to the arbitrary two random variables is generated to perform correlation analysis to obtain a target analysis result.

2. The method according to claim 1, characterized in that The preprocessing of the IoT perception data to obtain initial perception data includes: Using an isolation forest algorithm to remove outliers from the IoT perception data to obtain the initial perception data; and / or The Internet of Things perception data is enhanced by using a Lagrange interpolation method to obtain the initial perception data.

3. The method according to claim 2, characterized in that The method of using the isolation forest algorithm to remove outliers from the IoT perception data to obtain the initial perception data includes: Sorting the IoT perception data according to a time series, and determining the start and end time of each data source in the IoT perception data; Inputting the IoT sensing data and the start and end times into a preset isolation forest model to obtain an anomaly score for each data point in the data source; Determine the data point corresponding to the anomaly score being greater than the preset anomaly threshold as an outlier; The abnormal value is removed from the IoT perception data to obtain the initial perception data.

4. The method according to claim 2, characterized in that: The interpolating the IoT perception data using the Lagrange interpolation method to obtain the initial perception data includes: For the missing values ​​in the IoT sensing data, an estimated value is obtained by using the Lagrange interpolation method; The estimated value is inserted into the IoT perception data as a new data point to obtain the initial perception data.

5. The method according to claim 4, characterized in that The Lagrange interpolation polynomial in the Lagrange interpolation method is: Among them, x i is the horizontal coordinate of data point i in the IoT perception data; x j is the horizontal coordinate of data point j in the IoT perception data; y j is the ordinate of data point j in the IoT perception data; n is the number of data points in the IoT perception data.

6. The method according to claim 1, characterized in that The clustering of the initial perception data by the edge server to obtain multi-source data includes: Performing standardization processing on the initial perception data to obtain standard sample data; Determine K samples as the initial clustering centers; Clustering processing step: based on a preset distance function, determining the distance between the standard sample data and each of the initial cluster centers, and assigning each standard sample data to the class where the initial cluster center closest to the distance is located; Determine the mean of the standard sample data in the class as a new cluster center; Compare the error between the new cluster center and the initial cluster center, and if the error does not meet the preset condition, iterate the clustering processing step until the error meets the preset condition, so as to obtain the clustered multi-source data.

7. The method according to claim 1, characterized in that The determining, by the cloud server, the correlation coefficients corresponding to any two random variables in the multi-source data includes: Determine the Kendall rank correlation coefficient corresponding to any two random variables in the multi-source data; The correlation coefficient corresponding to the arbitrary two random variables is generated according to the Kendall rank correlation coefficient.

8. The method according to claim 7, characterized in that The relationship between the correlation coefficient and the Kendall rank correlation coefficient is: Among them, x is the independent variable, τ is the Kendall rank correlation coefficient, and θ is the correlation coefficient.

9. The method according to claim 1, characterized in that: The cumulative distribution function is: The probability density function is: Among them, x is the independent variable, x∈[0,1]; θ is the correlation coefficient, θ≠0, θ<0 means that the random variables u and v are positively correlated, θ>0 means that the random variables u and v are negatively correlated, and the closer θ is to 0, the more independent the random variables u and v tend to be.

10. A smart building multi-level collaborative IoT perception big data analysis system, characterized in that: The system comprises: A data acquisition unit, used to collect IoT perception data through sensors, and pre-process the IoT perception data to obtain initial perception data; A clustering processing unit, configured to cluster the initial perception data through an edge server to obtain multi-source data; A function generating unit, configured to determine, through a cloud server, a correlation coefficient corresponding to any two random variables in the multi-source data, and generate a cumulative distribution function and a probability density function corresponding to the any two random variables based on the correlation coefficient; The correlation analysis unit generates a joint probability function corresponding to any two random variables based on the cumulative distribution function and the probability density function to perform correlation analysis and obtain a target analysis result.

Images