Domestic sewage discharge monitoring method and system based on smart city

Abstract

The present invention relates to the technical field of data transmission, and in particular to a method and system for monitoring domestic sewage discharge based on a smart city, including: performing PCA analysis on monitoring data to obtain all principal component vectors, using the principal component vectors to reduce the dimension of the monitoring data to obtain lossless data and lossy data, performing lossy compression on the lossy data and transmitting it to a server together with the lossless data; obtaining the compression effectiveness when transmitting the monitoring data; using the similarity between the distribution of the monitoring data and the principal component vector to obtain the offset dimension and the stable dimension, according to the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device on the offset dimension and the stable dimension, reselecting a number of monitoring data from the monitoring data transmitted by each sensor device, performing PCA analysis on the reselected monitoring data, and reobtaining all principal component vectors of the monitoring data. The present invention improves the efficiency of the monitoring data when transmitting.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a method and system for monitoring domestic sewage discharge based on a smart city. Background Art

[0002] In the construction of smart cities, the monitoring of domestic sewage discharge has become an important part. By introducing the Internet of Things (IoT) technology, smart cities can realize online monitoring of domestic sewage discharge to ensure the health and safety of the urban environment. IoT technology realizes online monitoring of domestic sewage through a sensor network deployed at each sewage discharge point. These sensor devices can collect sewage monitoring data in real time. The collected monitoring data is transmitted to the server through a wireless network for data analysis and processing as well as visual monitoring. In the IoT sewage monitoring system, data transmission is a key link. However, with the increase in the number of sampling points and the complexity of the collection parameters, each sampling point generates a large amount of data in real time, which accumulates to form a huge data transmission demand. Traditional data transmission technology often shows low efficiency when facing large amounts of data. Large-scale data transmission occupies a large amount of network bandwidth and may affect the normal operation of other network services. In order to improve data transmission efficiency, compression technology has become an important means. Existing lossless compression algorithms, such as LZW (Lempel-Ziv-Welch), can reduce the amount of data without losing information, but their compression rate shows certain limitations when facing the rapidly growing data demand. Although lossy compression algorithms can significantly reduce the amount of data, they will lead to the loss of some information. Excessive information loss may lead to unreliable monitoring results, thus affecting the monitoring results. In summary, in the Internet of Things, the existing compression transmission method cannot achieve high transmission efficiency when transmitting domestic sewage monitoring data. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a domestic sewage discharge monitoring method and system based on a smart city.

[0004] The domestic sewage discharge monitoring method and system based on smart city of the present invention adopts the following technical solutions:

[0005] An embodiment of the present invention provides a method for monitoring domestic sewage discharge based on a smart city, the method comprising the following steps:

[0006] A plurality of sensor devices are used to collect monitoring data of domestic sewage from different sampling points. Each sensor device transmits the collected monitoring data to a server. After receiving the monitoring data from all sensor devices, the server selects a number of monitoring data from the monitoring data transmitted by each sensor device, performs PCA analysis on the selected monitoring data, obtains all principal component vectors of the monitoring data, and the server transmits all principal component vectors to the sensor device. Each sensor device uses all principal component vectors to reduce the dimension of the monitoring data. The data after the dimension reduction is recorded as lossless data, and the data corresponding to the dimension missing from the lossless data is recorded as lossy data. The lossy data is losslessly compressed and transmitted to the server together with the lossless data.

[0007] The compression effectiveness of each sensor device each time it transmits monitoring data is obtained according to the information loss amount before and after the lossy data is subjected to lossy compression, and the compression amount of the lossy data after the lossy compression;

[0008] By utilizing the similarity between the distribution of the monitoring data transmitted by each sensor device and the principal component vector, all principal component vectors are divided into an offset dimension and a stable dimension. According to the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device in the offset dimension and the stable dimension, several monitoring data are reselected from the monitoring data transmitted by each sensor device, and PCA analysis is performed on the reselected monitoring data to obtain all the principal component vectors of the monitoring data again.

[0009] Preferably, each sensor device uses all principal component vectors to reduce the dimension of the monitoring data, the data after the dimension reduction is recorded as lossless data, and the data corresponding to the missing dimension of the lossless data is recorded as lossy data, including the following specific steps:

[0010] Each principal component vector obtained in the PCA algorithm corresponds to an eigenvalue, and N principal component vectors with the largest eigenvalues ​​are obtained, and N projection values ​​of each monitoring data on the N principal component vectors are obtained, and the N-dimensional vector formed by the N projection values ​​is used as the lossless data of each monitoring data;

[0011] Obtain other principal component vectors other than the N principal component vectors with the largest eigenvalues, obtain the projection values ​​of each monitoring data on the other principal component vectors, record them as the data corresponding to the missing dimensions of the lossless data, and the vector composed of the data corresponding to the missing dimensions of the lossless data is recorded as the lossy data of each monitoring data;

[0012] N is a preset value.

[0013] Preferably, the lossy data is losslessly compressed, including the following specific steps:

[0014] For all values ​​in the lossy data, a closed interval consisting of the minimum and maximum values ​​is obtained, and the closed interval is equally divided into a number of sub-intervals. For each value in the lossy data, the sub-interval to which each value belongs is obtained, and the center point of the sub-interval to which it belongs is obtained, and each value is set equal to the value of the center point. Then, the Huffman coding algorithm is used to encode and compress all values ​​in the lossy data.

[0015] Preferably, the compression effectiveness is equal to the ratio of the compression amount to the information loss amount; the information loss amount is equal to the difference between the lossy data before the lossy compression and the lossy data after the lossy compression;

[0016] The specific steps for obtaining the compression amount are as follows:

[0017] The storage space occupied by the lossy data before lossy compression is a, and the storage space occupied by the lossy data after lossy compression is b. The compression amount is equal to (ab) / a.

[0018] Preferably, the method utilizes the similarity between the distribution of the monitoring data transmitted by each sensor device and the principal component vector to divide all the principal component vectors into an offset dimension and a stable dimension, and includes the following specific steps:

[0019] Perform PCA analysis on the monitoring data in the candidate data set of each sensor device to obtain all first vectors of each sensor device; use the KM matching algorithm to match all first vectors with all principal component vectors one by one;

[0020] For all first vectors of all sensor devices, each principal component vector is matched to a number of first vectors respectively;

[0021] The mean of the cosine similarities between each principal component vector and all the matched first vectors is recorded as the information representation offset of each principal component vector; the Otsu threshold segmentation algorithm is used to divide all principal component vectors into two categories according to the information representation offset, among which all the principal component vectors in the category with the largest average information representation offset are recorded as the offset dimension, and all the principal component vectors in the category with the smallest average information representation offset are recorded as the stable dimension.

[0022] Preferably, the reselecting of a plurality of monitoring data from the monitoring data transmitted by each sensor device according to the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device in the offset dimension and the stable dimension comprises the following specific steps:

[0023] The first vector that matches all offset dimensions is recorded as the offset matching dimension; the first vector that matches all stable dimensions is recorded as the stable matching dimension;

[0024] For any two monitoring data A and B in the candidate data set of each sensor device, obtain the information distribution feature Q1 of A and B in the offset dimension, and the information distribution feature Q2 of A and B in the offset matching dimension; obtain the absolute value of the difference between Q1 and Q2 and record it as the distribution difference Q between any two monitoring data A and B;

[0025] The mean of the similarity between the projection values ​​of A in the stable dimension and the stable matching dimension and the similarity between the projection values ​​of B in the stable dimension and the stable matching dimension is denoted as the average distribution stability M between any two monitoring data A and B;

[0026] According to the distribution difference Q, the average distribution stability M and the compression effectiveness, a number of monitoring data are reselected from the monitoring data transmitted by each sensor device.

[0027] Preferably, the reselecting of a plurality of monitoring data from the monitoring data transmitted by each sensor device according to the distribution difference Q, the average distribution stability M and the compression effectiveness comprises the following specific steps:

[0028] The selection validity of any two monitoring data is obtained based on the distribution difference Q, the average distribution stability M and the compression validity;

[0029] For all monitoring data in the candidate data set of each sensor device, the selection effectiveness of any two monitoring data is used as the metric distance, and all monitoring data are clustered using the metric distance. The mean of the selection effectiveness between all monitoring data in each category is recorded as the average selection effectiveness of each category; the mean of the compression effectiveness of all monitoring data in the candidate data set is recorded as C, and exp(-C)×K is taken as the screening number L for each sensor device; in the candidate data set of each sensor device, the monitoring data in the L categories with the largest average selection effectiveness are screened out; K represents the number of categories obtained, and exp() represents an exponential function with a natural constant as the base.

[0030] Preferably, the specific steps of obtaining the selection validity of any two monitoring data according to the distribution difference Q, the average distribution stability M and the compression validity are as follows:

[0031] The selection effectiveness of any two monitoring data is W=(1-y)×Q+y×M; where y represents the mean value of the information loss of the lossy data corresponding to any two monitoring data.

[0032] Preferably, the specific method for obtaining the candidate data set of each sensor device is as follows:

[0033] The compression validity obtained when each sensor device transmits monitoring data in turn constitutes a time series sequence, recorded as a validity sequence, and the monitoring data in the validity sequence whose compression validity change rate is greater than a preset second preset threshold and stored on the server is used as a candidate data set for each sensor device.

[0034] Another embodiment of the present invention provides a domestic sewage discharge monitoring system based on a smart city, the system includes a server and several sensor devices, the server and the sensor devices each include a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned domestic sewage discharge monitoring method based on a smart city when executing the computer program.

[0035] The beneficial effects of the technical solution of the present invention are:

[0036] Each sensor device uses all principal component vectors to reduce the dimension of the monitoring data. The data after dimension reduction is recorded as lossless data, and the data corresponding to the missing dimension of the lossless data is recorded as lossy data. The lossy data is lossily compressed and transmitted to the server together with the lossless data. Since the lossy data contains secondary information, although some data information is lost after lossy compression, the data information lost due to lossy compression does not affect the monitoring of domestic sewage, and the compressed lossy data has a smaller data volume when it is transmitted.

[0037] Further, the compression effectiveness of each sensor device each time the monitoring data is transmitted is obtained according to the amount of information loss before and after the lossy data is subjected to lossy compression, and the amount of compression of the lossy data after the lossy compression. The compression effectiveness is used to describe the data compression and information loss when the monitoring data is transmitted, and accurately describes the transmission efficiency of the transmission process. And further based on the compression effectiveness, and according to the distribution of the monitoring data transmitted from each sensor device in the offset dimension and the stable dimension, a number of monitoring data are reselected from the monitoring data transmitted by each sensor device, and the reselected monitoring data are subjected to PCA analysis to obtain all the principal component vectors of the monitoring data again. The present invention realizes the update of the principal component vector by re-reasonably screening the monitoring data (screening according to the distribution of the new monitoring data and the old principal component vector and the compression effectiveness of the old principal component vector when transmitting the monitoring data), avoids the principal component vector obtained from the old monitoring data from being unable to be applied to the subsequent transmission process of the monitoring data, ensures that the lossless data can accurately contain important information, and the lossy data only contains secondary information, so that the subsequent transmission of the monitoring data has a higher transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of the steps of a smart city-based domestic sewage discharge monitoring method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the method and system for monitoring domestic sewage discharge based on smart cities proposed by the present invention, its specific implementation method, structure, characteristics and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0041] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0042] The specific scheme of the domestic sewage discharge monitoring method and system based on smart city provided by the present invention is described in detail below with reference to the accompanying drawings.

[0043] See also Figure 1 , which shows a flowchart of a method for monitoring domestic sewage discharge based on a smart city provided by an embodiment of the present invention, the method comprising the following steps:

[0044] Step S001: Use multiple sensor devices to collect monitoring data of domestic sewage from different sampling points, and each sensor device transmits the collected monitoring data to a server.

[0045] Select several sampling points from all domestic sewage discharge points in the city, and place a sensor device at each sampling point. The sensor device includes multiple sensors for detecting multiple indicators of sewage, such as using a pH meter to detect the pH value of domestic sewage, using a conductivity meter to detect the conductivity of sewage, using a dissolved oxygen meter to detect the dissolved oxygen content of sewage, using a turbidity meter to detect the turbidity of sewage, and using a residual chlorine analyzer to detect the chlorine content. In other embodiments, sensors can be added or deleted on this basis.

[0046] In this embodiment, each sensor collects a piece of data every second, and each sensor device arranges all the collected data into a sequence every several moments (i.e. several seconds, for example, 5 seconds) to record the monitoring data of each sensor device. For example, a time series consisting of indicators such as pH value, conductivity, dissolved oxygen content, etc. within 5 seconds is obtained, and these time series are spliced ​​together from beginning to end to form the monitoring data of each sensor device.

[0047] Each sensor device transmits the obtained monitoring data to the server; this embodiment uses 5G network technology for transmission. After receiving the monitoring data transmitted by all sensor devices, the server stores the monitoring data in the server's database and plots the monitoring data on the display screen to facilitate online monitoring of various sewage indicators at each sampling point.

[0048] It should also be noted that the server only stores monitoring data within a certain period of time (for example, within one day), and expired monitoring data is deleted.

[0049] Step S002: After receiving the monitoring data from all sensor devices, the server selects some monitoring data from the monitoring data transmitted by each sensor device, performs PCA analysis on the selected monitoring data, obtains all principal component vectors of the monitoring data, and transmits all principal component vectors to the sensor devices.

[0050] Considering that there is redundant information in the monitoring data collected by each sensor device, for example, the changing trends between different indicator data are similar, and the data of the same indicator data at different times have the same changing rules, these will cause redundant information in the monitoring data. These redundant information not only wastes network resources when being transmitted (for example, it needs to occupy the server's traffic and bandwidth), but also when there is more redundant information, the transmission time is longer, which will consume more power of the sensor device.

[0051] In this embodiment, the server stores monitoring data from different sensor devices. It should be noted that each monitoring data is arranged into a sequence, and this sequence representation is regarded as a high-dimensional vector.

[0052] The server randomly selects a number (for example, 20) of monitoring data from the monitoring data of each sensor device, and performs PCA analysis on all the selected monitoring data. PCA is an existing algorithm. The PCA algorithm obtains a number of principal component vectors. According to the PCA algorithm, each principal component vector is a unit vector, which represents a main distribution direction of all monitoring data, and each principal component vector corresponds to an eigenvalue. The larger the eigenvalue, the more the direction of the principal component vector is the main distribution direction of all monitoring data. This embodiment uses all principal component vectors to describe the distribution of all monitoring data of all sensor devices (that is, the main distribution direction).

[0053] Whenever the server receives monitoring data transmitted from each sensor device, the server transmits all the principal component vectors and their corresponding eigenvalues ​​obtained above back to each sensor device.

[0054] Step S003: Each sensor device uses all principal component vectors to reduce the dimension of the monitoring data. The data after dimension reduction is recorded as lossless data, and the data corresponding to the dimension missing from the lossless data is recorded as lossy data. The lossy data is lossily compressed and transmitted to the server together with the lossless data.

[0055] After each sensor device receives all the principal component vectors, for the monitoring data collected again by the sensor device, the monitoring data corresponds to a vector, and the projection value of the vector on each principal component vector is obtained, and the N principal component vectors with the largest eigenvalues ​​are selected, and the N projection values ​​of the monitoring data on these N principal component vectors are obtained. The N-dimensional vector composed of these N projection values ​​is recorded as the dimensionality reduction result of the monitoring data. At this point, the monitoring data is reduced to N dimensions. The above process is well known, and its specific working principle will not be described in detail in this embodiment. This embodiment is described using N=10 as an example. In other embodiments, N can be set to other values, and this embodiment does not specifically limit it.

[0056] In this embodiment, the above-mentioned dimensionality reduction result is recorded as lossless data. Further, other principal component vectors other than the N principal component vectors with the largest eigenvalues ​​are obtained, assuming that there are N0 other principal component vectors, and N0 projection values ​​of the monitoring data on these N0 other principal component vectors are obtained. These N0 projection values ​​are the dimensional data missing from the lossless data, and the N0-dimensional vectors formed by these N0 projection values ​​are recorded as the lossy data of the monitoring data.

[0057] The above lossless data contains the main sewage information of the monitoring data, and the lossy data contains the secondary information. In this embodiment, the lossy data containing the secondary information is losslessly compressed to obtain compressed lossy data. In this embodiment, the compressed lossy data and the lossless data are transmitted to the server together.

[0058] In other embodiments, a lossless compression algorithm (such as the LZW algorithm) may be used to compress the lossless data and then transmit the compressed lossy data to the server together.

[0059] In the above process, since the lossy data contains secondary information, after lossy compression, although a certain amount of data information is lost, the data information lost due to lossy compression does not affect the monitoring of domestic sewage. At the same time, the compressed lossy data has a smaller data volume during transmission, avoiding excessive waste of network resources and saving power consumption of sensor equipment during transmission.

[0060] As an example, lossy compression is performed on lossy data to obtain compressed lossy data, including the following steps:

[0061] Get each value in the lossy data, get the closed interval formed by the minimum and maximum values ​​therein, divide the closed interval into N1 sub-intervals, and for each value in the lossy data, get the sub-interval to which each value belongs, get the center point of the sub-interval, set each value equal to the value of the center point, that is, take the value of the center point of the sub-interval as each value in the lossy data, and then use the Huffman coding algorithm to encode and compress all the values ​​in the lossy data to obtain compressed lossy data.

[0062] In this embodiment, the method for obtaining N1 is: obtaining the difference between the maximum value and the minimum value, and taking the ratio of the difference to 10 as N1 (needs to be rounded up). In other embodiments, N1 may be set to other values, which are not specifically limited in this embodiment.

[0063] When the server receives the lossy data and lossless data after lossy compression transmitted by each device, it first decompresses the lossy data after lossy compression to obtain the lossy data, and uses all the principal component vectors to restore the lossy data and lossless data into the monitoring data transmitted by each device, and then stores the monitoring data and displays it on the display screen.

[0064] As an example, all principal component vectors are used to restore the lossy data and the lossless data into monitoring data transmitted by each device, including the following method:

[0065] The lossless data and the lossy data are concatenated together to form a vector V0. The matrix composed of all principal component vectors is denoted as D0 (each row in D0 is a principal component vector), and the unit matrix with the same rows and columns as D0 is denoted as D1. Assuming that the vector corresponding to the monitoring data is denoted as V1, then D0*V0=D1*V1 is satisfied. By solving V1 according to this equation, the monitoring data transmitted by each device can be obtained.

[0066] The above process involves the technology of using a matrix to perform spatial transformation on a vector, which is a well-known technology in PCA, and the specific working principle thereof will not be described in detail in this embodiment.

[0067] Step S004: Obtain the compression effectiveness of each sensor device each time it transmits monitoring data according to the information loss amount before and after the lossy data is subjected to lossy compression, and the compression amount of the lossy data after the lossy compression.

[0068] The values ​​of the lossy data before lossy compression and the lossy data after lossy compression are different, which means that there is information loss before and after lossy compression, and the amount of information loss is equal to the difference between the lossy data before lossy compression and the lossy data after lossy compression. The greater the amount of information loss, the more different the values ​​of the lossy data before and after lossy compression, and the more sewage monitoring information is lost.

[0069] As an example, a method for obtaining the difference between lossy data before lossy compression and lossy data after lossy compression is as follows:

[0070] The lossy data before lossy compression is represented as a vector, and the lossy data after lossy compression is also represented as a vector. The modulus of the means of the two vectors is obtained, and the ratio of the Euclidean distance of the two vectors to the modulus is recorded as the difference between the lossy data before lossy compression and the lossy data after lossy compression.

[0071] The storage space occupied by the lossy data before lossy compression is a, and the storage space occupied by the lossy data after lossy compression is b. The compression amount is equal to (ab) / a. The greater the compression amount, the more network resources can be saved during transmission.

[0072] In this embodiment, the ratio of the compression amount to the information loss amount is recorded as the compression effectiveness of each sensor device. In particular, if the information loss amount is less than or equal to 0.1, then the information loss amount is set to be equal to 0.1 to avoid the dimension of the compression effectiveness being too large.

[0073] The compression effectiveness of each sensor device is used to describe whether the monitoring data has a high transmission efficiency when transmitted according to the method in step S003. When the compression effectiveness is large, it means that network resources can be saved more significantly with less information loss, and the transmission efficiency is high at this time. When the compression effectiveness is small, it means that network resources cannot be saved more significantly when more information is lost, and the transmission efficiency is low at this time.

[0074] At this point, after each sensor device transmits according to the method in step S003, the server obtains the compression validity of each sensor device each time it transmits monitoring data (that is, after each sensor device calculates the compression validity, it also transmits the compression validity to the server).

[0075] Step S005: using the similarity between the distribution of the monitoring data transmitted from each sensor device and the principal component vector, all principal component vectors are divided into an offset dimension and a stable dimension.

[0076] It should be noted that as the sensor device collects monitoring data, the characteristics reflected by the monitoring data collected by the same sensor device in different time periods, and the monitoring data collected by different sensor devices in the same time period are different and not static. Therefore, the principal component vector obtained according to the screened monitoring data in step S002 cannot be adapted to the transmission process of all subsequent monitoring data, which will cause the lossless data in step S003 to be unable to accurately contain important information, and the lossy data to be unable to contain only secondary information, which will further cause the lossy data to be difficult to obtain a high transmission efficiency even after lossy compression.

[0077] As the sensor device continuously transmits monitoring data (transmitted according to the method in step S003), when there is a sensor device and the compression validity of the sensor device is less than the first preset threshold th1, it is necessary to re-obtain all principal component vectors. This embodiment is described by taking th1=2 as an example, and other embodiments may set th1 to other values, which are not specifically limited in this embodiment.

[0078] Before re-obtaining all principal component vectors, all principal component vectors are divided into offset dimensions and stable dimensions by using the similarity between the distribution of monitoring data transmitted by each sensor device and the principal component vector. The specific method is as follows:

[0079] In the above process, each sensor device can obtain a compression validity after transmitting a monitoring data each time. The compression validity obtained when each sensor device transmits the monitoring data in turn constitutes a time series, which is recorded as a validity sequence. The monitoring data in the validity sequence whose compression validity change rate is greater than the second preset threshold th2 and stored on the server is used as the candidate data set of each sensor device. This embodiment is described by taking th2=0.3 as an example. In other embodiments, th2 can be set to other values, which are not specifically limited in this embodiment.

[0080] As an example, the method for obtaining the compression validity change rate in the validity sequence is:

[0081] In the validity sequence, the i-th compression validity is recorded as f(i), the i-1-th compression validity is recorded as f(i-1), and the ratio of |f(i)-f(i-1)| to f(i) is recorded as the i-th compression validity change rate. In this embodiment, the first compression validity change rate is equal to the second compression validity change rate.

[0082] The compression effectiveness corresponding to the monitoring data in the candidate data set changes rapidly (that is, the compression effectiveness change rate is large), indicating that new feature information that is different from the previous ones appears in these monitoring data (such as new data distribution). These feature information may be important information, but appear in lossy data, which makes it difficult to efficiently transmit the lossy data (the large amount of information loss leads to the difficulty in efficient transmission); it may also be secondary information, but some of this secondary information is not included in the lossy data, which also makes it difficult to efficiently transmit (the small amount of compression leads to the difficulty in efficient transmission). In summary, since the lossy data cannot accurately describe the secondary information of the monitoring data, the lossy data is difficult to transmit efficiently after lossy compression.

[0083] Further, PCA analysis is performed on all monitoring data in the candidate data set of each sensor device to obtain all principal component vectors of each sensor device, which are recorded as the first vector of each sensor. The difference between the first vector and the principal component vector obtained in step S002 is that, on the one hand, the principal component vector obtained in step S002 describes the distribution of old monitoring data in the past (for example, the main distribution direction of the old monitoring data), and the first vector describes the distribution of newly acquired monitoring data of each sensor device, and does not include the distribution of monitoring data of other sensor devices other than each sensor device, so it can relatively accurately describe the characteristics of sewage discharge at each sampling point; on the other hand, the difference is that the monitoring data in the candidate data set is lossy (that is, it is lossy compared with the monitoring data directly collected by the sensor device or previously screened); the above difference makes the first vector different from the principal component vector obtained in step S002.

[0084] For all first vectors of each sensor device, the KM matching algorithm is used to match all first vectors with all principal component vectors obtained in step S002 one by one, that is, each principal component vector obtained in step S002 is matched with a most similar first vector.

[0085] For all first vectors of all sensor devices, each principal component vector obtained in step S002 matches a plurality of first vectors (the plurality of first vectors are from each sensor device).

[0086] The mean of the cosine similarities between each principal component vector and all the first vectors it matches is recorded as the information representation offset of each principal component vector. As the server continuously receives monitoring data from different sensor devices, the principal component vector is no longer suitable for describing the distribution of the monitoring data of the current sensor device. The information representation offset represents the difference between the distribution of the monitoring data described by the principal component vector and the distribution of the current monitoring data.

[0087] The Otsu threshold segmentation algorithm is used to divide all principal component vectors obtained in step S002 into two categories according to the information representation offset, wherein all principal component vectors in a category with the largest average information representation offset are recorded as the offset dimension, and all principal component vectors in a category with the smallest average information representation offset are recorded as the stable dimension.

[0088] The offset dimension and the stable dimension describe the distribution of the monitoring data from two aspects respectively. Specifically, compared with the distribution of the monitoring data newly received by the server (that is, the candidate data set of all the above-mentioned sensor devices), the data distribution described by the part of the principal component vectors obtained in S002 has deviated, and the offset dimension represents this part of the principal component vectors. Compared with the distribution of the monitoring data newly received by the server, the data distribution described by the other part of the principal component vectors obtained in S002 has no obvious deviation, and the stable dimension represents the other part of the principal component vectors, indicating that the monitoring data newly collected by all sensor devices still contains part of the data distribution of the old monitoring data (that is, the monitoring data selected in step S002), that is, the stable dimension represents the stable and unchanging distribution characteristics of the monitoring data.

[0089] Step S006: based on the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device in the offset dimension and the stable dimension, reselect a number of monitoring data from the monitoring data transmitted by each sensor device, perform PCA analysis on the reselected monitoring data, and re-obtain all principal component vectors of the monitoring data.

[0090] For any two monitoring data A and B in the candidate data set of each sensor device, obtain the projection of A on all offset dimensions, and the vector formed by these projections is recorded as F1(A); obtain the projection of B on all offset dimensions, and the vector formed by these projections is recorded as F1(B); obtain the Euclidean distance between F1(A) and F1(B), and record it as the information distribution feature Q1 between any two monitoring data A and B and in the offset dimension.

[0091] The first vector of each sensor device that matches all offset dimensions is recorded as the offset matching dimension. The projection of A on all offset matching dimensions is obtained, and the vector formed by these projections is recorded as F2(A). The projection of B on all offset matching dimensions is obtained, and the vector formed by these projections is recorded as F2(B). The Euclidean distance between F2(A) and F2(B) is obtained and recorded as the information distribution feature Q2 between any two monitoring data A and B in the offset matching dimension.

[0092] The absolute value of the difference between Q1 and Q2 is obtained, which is recorded as the distribution difference Q between any two monitoring data A and B. It describes the change in the relative distribution between any two monitoring data when the monitoring data changes over time.

[0093] The first vector of each sensor device that matches all stable dimensions is recorded as a stable matching dimension.

[0094] Get the projections of A on all stable dimensions, and the vectors formed by these projections are recorded as F3(A). Get the projections of A on all stable matching dimensions, and the vectors formed by these projections are recorded as F4(A). The cosine similarity between F3(A) and F4(A) is recorded as the distribution stability of A. Get the projections of B on all stable dimensions, and the vectors formed by these projections are recorded as F3(B). Get the projections of B on all stable matching dimensions, and the vectors formed by these projections are recorded as F4(B). The cosine similarity between F3(B) and F4(B) is recorded as the distribution stability of B. The mean of the distribution stability of A and the distribution stability of B is recorded as the average distribution stability M between any two monitoring data A and B. The smaller the average distribution stability, the greater the difference between A and B under the stable and unchanged distribution characteristics of the monitoring data, indicating that A and B do not obviously contain the distribution of the old monitoring data. The greater the average distribution stability, the more obvious the similarity between A and B under the stable and unchanged distribution characteristics of the monitoring data, indicating that A and B contain the distribution of the old monitoring data.

[0095] The selection validity of any two monitoring data A and B is W=(1-y)×Q+y×M. The greater the selection validity, the more it means that the monitoring data A and B have introduced new distribution characteristics while containing the distribution of the old monitoring data (that is, when M is large). At this time, it is more necessary to screen A and B from the candidate data set. When the selection validity is smaller, it means that the monitoring data A and B do not obviously contain the distribution of the old monitoring data, and do not introduce new distribution characteristics. At this time, there is no need to screen A and B from the candidate data set.

[0096] Where y represents the mean of the information loss of the lossy data corresponding to A and B. The larger the y is, the more information A and B lose during transmission. At this time, we pay more attention to the size of the average distribution stability M. When screening monitoring data based on the selection effectiveness W, we pay more attention to whether the screened monitoring data contains more distribution of old monitoring data, so as to avoid the problem that the distribution difference Q cannot reliably describe the newly introduced distribution characteristics due to excessive information loss during transmission, thereby making the selection effectiveness W inaccurate.

[0097] The smaller y is, the less information A and B lose during transmission. At this time, we pay more attention to the size of the distribution difference Q. When screening monitoring data based on the selection effectiveness W, we pay more attention to the screened monitoring data with obvious changes in data distribution.

[0098] Methods for screening monitoring data based on selection effectiveness include:

[0099] For all monitoring data in the candidate data set of each sensor device, the selection validity W of any two monitoring data is used as the metric distance, and K-Means clustering is performed on all monitoring data using the metric distance. The number of clusters is set to K, and each sensor device corresponds to K categories. The mean of the selection validity W between all monitoring data in each category is recorded as the average selection validity of each category. In this embodiment, K is: one-fourth of the number of all monitoring data in the candidate data set of each sensor device (need to be rounded up).

[0100] The mean compression effectiveness of all monitoring data in the candidate data set of each sensor device is denoted as C, and exp(-C)×K is used as the screening number L for each sensor device (rounded up). That is, the smaller the compression effectiveness, the more monitoring data needs to be screened out to ensure that each sensor device can be efficiently transmitted later.

[0101] Among the candidate data sets of each sensor device, the monitoring data in L categories with the largest average selection effectiveness are screened out.

[0102] Finally, all monitoring data filtered by all sensor devices are subjected to PCA analysis to retrieve all principal component vectors. All principal component vectors retrieved are suitable for describing the distribution of subsequent monitoring data and can ensure the efficiency of monitoring data in the subsequent transmission process.

[0103] Next, based on all the principal component vectors that have been retrieved, subsequent transmission is performed according to the method of step S003. During the subsequent transmission process, all the principal component vectors are continuously updated according to the methods of steps S004 to S006.

[0104] It should be noted that in other embodiments, the method of steps S002 to S006 is no longer used to transmit monitoring data at regular time intervals (for example, every 2 hours or half a day), but the monitoring data of each sensor device is directly transmitted. Although the transmission efficiency of the monitoring data is not high within a certain time period, lossless monitoring results can be obtained by this method.

[0105] In addition, when the server does not store monitoring data for a certain period of time (for example, one day), the monitoring data of each sensor device is directly transmitted, and the above-mentioned steps S002 to S006 are no longer used to transmit the monitoring data, thereby avoiding the problem that the server has too little monitoring data and cannot perform accurate PCA analysis.

[0106] In another embodiment, a domestic sewage discharge monitoring system based on a smart city is provided, which includes a server and several sensor devices. The server and the sensor devices both include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods in steps S001 to S006 described above are implemented.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring domestic sewage discharge based on smart cities, characterized in that: The method comprises the following steps: A plurality of sensor devices are used to collect monitoring data of domestic sewage from different sampling points. Each sensor device transmits the collected monitoring data to a server. After receiving the monitoring data from all sensor devices, the server selects a number of monitoring data from the monitoring data transmitted by each sensor device, performs PCA analysis on the selected monitoring data, obtains all principal component vectors of the monitoring data, and the server transmits all principal component vectors to the sensor device. Each sensor device uses all principal component vectors to reduce the dimension of the monitoring data. The data after the dimension reduction is recorded as lossless data, and the data corresponding to the dimension missing from the lossless data is recorded as lossy data. The lossy data is losslessly compressed and transmitted to the server together with the lossless data. The compression effectiveness of each sensor device each time it transmits monitoring data is obtained according to the information loss amount before and after the lossy data is subjected to lossy compression, and the compression amount of the lossy data after the lossy compression; By using the similarity between the distribution of the monitoring data transmitted by each sensor device and the principal component vector, all principal component vectors are divided into an offset dimension and a stable dimension. According to the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device in the offset dimension and the stable dimension, a number of monitoring data are reselected from the monitoring data transmitted by each sensor device, and PCA analysis is performed on the reselected monitoring data to obtain all principal component vectors of the monitoring data again. The method utilizes the similarity between the distribution of the monitoring data transmitted from each sensor device and the principal component vector to divide all the principal component vectors into an offset dimension and a stable dimension, and includes the following specific steps: Perform PCA analysis on the monitoring data in the candidate data set of each sensor device to obtain all first vectors of each sensor device; use the KM matching algorithm to match all first vectors with all principal component vectors one by one; For all first vectors of all sensor devices, each principal component vector is matched to a number of first vectors respectively; The mean of the cosine similarities between each principal component vector and all the matched first vectors is recorded as the information representation offset of each principal component vector; all principal component vectors are divided into two categories according to the information representation offset using the Otsu threshold segmentation algorithm, where all principal component vectors in a category with the largest average information representation offset are recorded as the offset dimension, and all principal component vectors in a category with the smallest average information representation offset are recorded as the stable dimension; The specific steps of reselecting a plurality of monitoring data from the monitoring data transmitted by each sensor device according to the compression effectiveness and the distribution of the monitoring data transmitted by each sensor device in the offset dimension and the stable dimension are as follows: The first vector that matches all offset dimensions is recorded as the offset matching dimension; the first vector that matches all stable dimensions is recorded as the stable matching dimension; For any two monitoring data A and B in the candidate data set of each sensor device, obtain the information distribution feature Q1 of A and B in the offset dimension, and the information distribution feature Q2 of A and B in the offset matching dimension; obtain the absolute value of the difference between Q1 and Q2 and record it as the distribution difference Q between any two monitoring data A and B; The mean of the similarity between the projection values ​​of A in the stable dimension and the stable matching dimension and the similarity between the projection values ​​of B in the stable dimension and the stable matching dimension is denoted as the average distribution stability M between any two monitoring data A and B; Reselect a number of monitoring data from the monitoring data transmitted by each sensor device according to the distribution difference Q, the average distribution stability M and the compression effectiveness; The specific steps of reselecting a number of monitoring data from the monitoring data transmitted by each sensor device according to the distribution difference Q, the average distribution stability M and the compression effectiveness are as follows: The selection validity of any two monitoring data is obtained based on the distribution difference Q, the average distribution stability M and the compression validity; For all monitoring data in the candidate data set of each sensor device, the selection effectiveness of any two monitoring data is used as the metric distance, and all monitoring data are clustered using the metric distance. The mean of the selection effectiveness between all monitoring data in each category is recorded as the average selection effectiveness of each category; the mean of the compression effectiveness of all monitoring data in the candidate data set is recorded as C, and exp(-C)×K is used as the screening number L of each sensor device; in the candidate data set of each sensor device, the monitoring data in the L categories with the largest average selection effectiveness are screened out; where K represents the number of categories obtained, and exp() represents an exponential function with a natural constant as the base; The specific steps of obtaining the selection validity of any two monitoring data according to the distribution difference Q, the average distribution stability M and the compression validity are as follows: The selection effectiveness of any two monitoring data is W=(1-y)×Q+y×M; where y represents the mean value of the information loss of the lossy data corresponding to any two monitoring data.

2. According to the method for monitoring domestic sewage discharge based on smart city according to claim 1, it is characterized in that: Each sensor device uses all principal component vectors to reduce the dimension of the monitoring data, and the data after the dimension reduction is recorded as lossless data, and the data corresponding to the missing dimension of the lossless data is recorded as lossy data. The specific steps included are as follows: Each principal component vector obtained in the PCA algorithm corresponds to an eigenvalue, and N principal component vectors with the largest eigenvalues ​​are obtained, and N projection values ​​of each monitoring data on the N principal component vectors are obtained, and the N-dimensional vector formed by the N projection values ​​is used as the lossless data of each monitoring data; Obtain other principal component vectors other than the N principal component vectors with the largest eigenvalues, obtain the projection values ​​of each monitoring data on the other principal component vectors, record them as the data corresponding to the missing dimensions of the lossless data, and the vector composed of the data corresponding to the missing dimensions of the lossless data is recorded as the lossy data of each monitoring data; N is a preset value.

3. The method for monitoring domestic sewage discharge based on smart city according to claim 1 is characterized in that: The lossy data is losslessly compressed, and the specific steps include the following: For all values ​​in the lossy data, a closed interval consisting of the minimum and maximum values ​​is obtained, and the closed interval is equally divided into a number of sub-intervals. For each value in the lossy data, the sub-interval to which each value belongs is obtained, and the center point of the sub-interval to which it belongs is obtained, and each value is set equal to the value of the center point. Then, the Huffman coding algorithm is used to encode and compress all values ​​in the lossy data.

4. The method for monitoring domestic sewage discharge based on smart city according to claim 1 is characterized in that: The compression effectiveness is equal to the ratio of the compression amount to the information loss amount; the information loss amount is equal to the difference between the lossy data before the lossy compression and the lossy data after the lossy compression; The specific steps for obtaining the compression amount are as follows: The storage space occupied by the lossy data before lossy compression is a, and the storage space occupied by the lossy data after lossy compression is b. The compression amount is equal to (ab) / a.

5. The method for monitoring domestic sewage discharge based on smart city according to claim 1 is characterized in that: The specific method for obtaining the candidate data set of each sensor device is as follows: The compression validity obtained when each sensor device transmits monitoring data in turn constitutes a time series sequence, recorded as a validity sequence, and the monitoring data in the validity sequence whose compression validity change rate is greater than a preset second preset threshold and stored on the server is used as a candidate data set for each sensor device.

6. A domestic sewage discharge monitoring system based on a smart city, the system comprising a server and several sensor devices, the server and the sensor devices each comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of any smart city-based domestic sewage discharge monitoring method as claimed in claims 1-5 are implemented.

Images