A data compression storage method for ultrasonic water meter of Internet of Things

Abstract

The present invention relates to the technical field of data compression, and in particular to a data compression and storage method for an ultrasonic water meter in the Internet of Things. The method obtains N-dimensional monitoring data collected by the ultrasonic water meter within a preset time period for any ultrasonic water meter, constructs an N*M two-dimensional matrix, and obtains the data fluctuation degree of each ultrasonic water meter; performs deep stacking on the two-dimensional matrices of all ultrasonic water meters in the order of all data fluctuation degrees from small to large to obtain a stacking matrix, and divides the stacking matrix into two sub-stacking matrices; performs normalization processing on each element in each sub-stacking matrix to obtain a normalized matrix, obtains a data sequence to be compressed according to an adaptive compression path of each normalized matrix, performs run-length encoding on the data sequence to be compressed to obtain compressed data, and the repeatability of the data sequence to be compressed obtained based on the adaptive compression path is higher, thereby improving the compression efficiency of the multi-dimensional monitoring data of the ultrasonic water meter.

Description

Technical Field

[0001] The present invention relates to the technical field of data compression, and in particular to a data compression and storage method for an ultrasonic water meter of the Internet of Things. Background Art

[0002] Ultrasonic water meter is an advanced water flow meter that can provide accurate water flow data. It uses ultrasonic technology to measure the speed and volume of water flowing through the pipe, and then transmits the measured water flow data to the centralized management system through the Internet of Things technology (such as NB-IoT, LoRa, Sigfox, GPRS, etc.) to achieve remote monitoring and data analysis. The historical data collected by the ultrasonic water meter needs to be stored for data retrieval and analysis. However, due to the high accuracy of the ultrasonic water meter's measurement data and the extremely large amount of data when the Internet of Things is used to centrally store the data of multiple ultrasonic water meters, an efficient data compression storage method is required.

[0003] The existing technology can usually use the run-length encoding compression algorithm (RLE) to compress the collected data of the ultrasonic water meter. Among them, run-length encoding reduces the data storage space by encoding continuous identical data, and has better compression efficiency for data with a lot of continuous repeated content. However, the collected data of the ultrasonic water meter contains a large amount of similar but non-repeated redundant data, which leads to poor compression efficiency of the collected data of the ultrasonic water meter using run-length encoding.

[0004] Therefore, how to improve the compression efficiency of ultrasonic water meter data collected by run-length encoding has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a data compression and storage method for an Internet of Things ultrasonic water meter to solve the problem of how to improve the compression efficiency of the collected data of the ultrasonic water meter using run-length encoding.

[0006] An embodiment of the present invention provides a data compression and storage method for an Internet of Things ultrasonic water meter, the method comprising the following steps:

[0007] For any ultrasonic water meter, obtain N-dimensional monitoring data collected by the ultrasonic water meter within a preset period of time, and construct the N-dimensional monitoring data collected within the preset period of time into an N*M two-dimensional matrix, and obtain the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset period of time;

[0008] Obtaining the data fluctuation degree of each ultrasonic water meter, deeply stacking the two-dimensional matrices of all ultrasonic water meters in the order of all data fluctuation degrees from small to large to obtain a stacked matrix, and dividing the stacked matrix into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter;

[0009] Normalizing each element in each of the sub-stacked matrices to obtain a corresponding normalized matrix, and obtaining an adaptive compression path for each of the normalized matrices according to the neighborhood similarity of each data in each of the normalized matrices and the overall data volatility;

[0010] According to the adaptive compression path of each normalized matrix, a data sequence to be compressed is obtained, and run-length encoding is performed on the data sequence to be compressed to obtain corresponding compressed data.

[0011] Preferably, obtaining the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset time period includes:

[0012] For any dimension of the N-dimensional monitoring data collected within the preset time period, a monitoring data change curve under the dimension is constructed, all peak points of the monitoring data change curve are obtained, and for any peak point, the value of the peak point is normalized to obtain a corresponding first normalized value, and the absolute value of the slope difference between the peak point and its left and right adjacent coordinate points is obtained, and the absolute value of the slope difference is normalized to obtain a corresponding second normalized value;

[0013] Obtaining a first time interval between the peak point and its left adjacent peak point, and a second time interval between the peak point and its right adjacent peak point, obtaining the inverse of a sum of the first time interval and the second time interval, and obtaining a product of the first normalized value, the second normalized value, and the inverse of the sum as a fluctuation characteristic value of the peak point;

[0014] The accumulated value of the fluctuation characteristic values ​​of all peak points of the monitoring data change curve is obtained as the data fluctuation index under the dimension, and the accumulated value of the data fluctuation index of each dimension of the N-dimensional monitoring data collected within the preset time period is used as the data fluctuation degree of the ultrasonic water meter.

[0015] Preferably, the stacking matrix is ​​divided into two sub-stacking matrices according to the data fluctuation degree of each ultrasonic water meter, including:

[0016] Normalizing the data fluctuation degree of each ultrasonic water meter respectively to obtain a corresponding normalized value, and obtaining a preset normalized threshold value. If the normalized value corresponding to any ultrasonic water meter is less than or equal to the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with small data fluctuation. On the contrary, if the normalized value corresponding to any ultrasonic water meter is greater than the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with large data fluctuation.

[0017] The two-dimensional matrices with small data fluctuations in the stacked matrix are composed into a first sub-stacked matrix, and the two-dimensional matrices with large data fluctuations in the stacked matrix are composed into a second sub-stacked matrix.

[0018] Preferably, the step of obtaining the adaptive compression path of each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility comprises:

[0019] For the normalized matrix corresponding to the first sub-stacked matrix, construct the three-dimensional space of the normalized matrix to obtain three spatial planes, for any spatial plane, obtain the absolute value of the data difference between each data in the spatial plane and each neighboring data within its preset neighborhood range, obtain the accumulated value of the absolute value of the data difference of each data in the spatial plane, use an exponential function with a natural constant as the base to inversely normalize the sum of the accumulated values ​​of the absolute values ​​of the data differences of all data, and use the result obtained as the overall similarity eigenvalue of the spatial plane;

[0020] According to the overall similarity eigenvalue of each of the spatial planes, the spatial plane corresponding to the largest overall similarity eigenvalue is obtained as the traversal plane, and a horizontal Z-shaped traversal method is used on the traversal plane as the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix.

[0021] Preferably, the step of obtaining the adaptive compression path of each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility comprises:

[0022] For any ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, the Pearson correlation coefficient between the normalized data sequences of every two dimensions is calculated according to the normalized data sequence of each dimension respectively corresponding to the N-dimensional monitoring data of the ultrasonic water meter in the normalized matrix, and the difference between each Pearson correlation coefficient and a constant 1 is obtained as the multi-dimensional correlation eigenvalue of the ultrasonic water meter by adding a difference cumulative value;

[0023] According to the multi-dimensional correlation eigenvalues ​​of each ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, a cumulative value of the multi-dimensional correlation eigenvalues ​​is obtained, and the cumulative value of the multi-dimensional correlation eigenvalues ​​is negatively mapped by using an exponential function with a natural constant as the base, and the obtained mapping value is used as the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix;

[0024] According to the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix, an adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained.

[0025] Preferably, obtaining the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix according to the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix includes:

[0026] Obtaining a preset overall data association index threshold, if the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​greater than or equal to the overall data association index threshold, using a longitudinal Z-shaped traversal method on a preset traversal plane as an adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix;

[0027] If the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​less than the overall data association index threshold, the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained, and a horizontal Z-shaped traversal method is used on the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix as the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix.

[0028] Preferably, the step of obtaining the to-be-compressed data sequence according to the adaptive compression path of each normalized matrix comprises:

[0029] For any normalized matrix, the data in the normalized matrix are traversed in sequence according to the adaptive compression path of the normalized matrix to obtain a traversed data sequence, and the traversed data sequence is subjected to a first-order difference to obtain a differential sequence, and the differential sequence is placed after the first data in the traversed data sequence to form a data sequence to be compressed.

[0030] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0031] The present invention aims at any ultrasonic water meter, obtains N-dimensional monitoring data collected by the ultrasonic water meter within a preset time period, and constructs the N-dimensional monitoring data collected within the preset time period into an N*M two-dimensional matrix, and obtains the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset time period; obtains the data fluctuation degree of each ultrasonic water meter, and deeply stacks the two-dimensional matrices of all ultrasonic water meters in the order of all data fluctuation degrees from small to large to obtain a stacked matrix, and divides the stacked matrix into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter; normalizes each element in each sub-stacked matrix to obtain a corresponding normalized matrix, and obtains the adaptive compression path of each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility; obtains a data sequence to be compressed according to the adaptive compression path of each normalized matrix, and runs the data sequence to be compressed to obtain corresponding compressed data. Among them, by performing data fluctuation analysis on the N-dimensional monitoring data of multiple ultrasonic water meters connected to the Internet of Things, the two-dimensional matrices of all ultrasonic water meters are deeply stacked to obtain a stacked matrix, that is, a multi-dimensional space is constructed, and then based on the multi-dimensional space where the stacked matrix is ​​located, the stacked matrix is ​​divided into two sub-stacked matrices with large data volatility and small data volatility, so as to be able to subsequently obtain adjacent compression paths with high similarity in the form of differences, so that the possibility of the data after differentiation being repeated data is higher. Therefore, data similarity or overall data volatility analysis is performed on each sub-stacked matrix to obtain an adaptive compression path for each sub-stacked matrix, so that the repeatability in the data sequence to be compressed obtained based on the adaptive compression path is higher, thereby improving the compression efficiency of run-length encoding of the data sequence to be compressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 This is a method flow chart of a data compression and storage method for an Internet of Things ultrasonic water meter provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0035] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0036] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0037] See also Figure 1 , is a method flow chart of a data compression and storage method for an Internet of Things ultrasonic water meter provided in Embodiment 1 of the present invention, such as Figure 1 As shown, the method may include:

[0038] Step S101, for any ultrasonic water meter, obtain N-dimensional monitoring data collected by the ultrasonic water meter within a preset time period, and construct the N-dimensional monitoring data collected within the preset time period into an N*M two-dimensional matrix, and obtain the data fluctuation degree of the ultrasonic water meter according to the data changes of the N-dimensional monitoring data collected within the preset time period.

[0039] After collecting multi-dimensional monitoring data, each ultrasonic water meter distributed in multiple locations transmits the collected monitoring data to a centralized management system through the Internet of Things technology. In the embodiment of the present invention, 24 hours is a preset period, and for any ultrasonic water meter, the data monitored by the ultrasonic water meter within 24 hours is collected through a preset sampling frequency. The main data types collected are instantaneous flow, water temperature and water pressure, that is, the instantaneous flow, water temperature and water pressure monitored by the ultrasonic water meter within 24 hours are collected, and one sampling moment collects one instantaneous flow, one water temperature and one water pressure, that is, N=3, and then all the instantaneous flow, water temperature and water pressure collected within 24 hours are formed into a 3*M two-dimensional matrix, the first line of the two-dimensional matrix is ​​the instantaneous flow, the second line is the water temperature, and the third line is the water pressure. The columns of the two-dimensional matrix refer to the M sampling moments within 24 hours. Preferably, the sampling frequency is set to 1 minute, corresponding to M=1440.

[0040] Similarly, the data of each ultrasonic level monitored within 24 hours is obtained to construct a corresponding 3*M two-dimensional matrix, that is, one ultrasonic water meter corresponds to one two-dimensional matrix.

[0041] Run-length encoding has a better compression effect on data with a large degree of repetition, and the purpose of the embodiment of the present invention is to improve the compression efficiency of run-length encoding of data in the two-dimensional matrix of all ultrasonic water meters by converting the data distribution in the two-dimensional matrix. Therefore, for any ultrasonic water meter, the data fluctuation degree of the ultrasonic water meter is obtained according to the data change of the three-dimensional monitoring data collected within a preset period of time. The method for obtaining the data fluctuation degree of the ultrasonic water meter is:

[0042] For any dimension of the N-dimensional monitoring data collected within the preset time period, a monitoring data change curve under the dimension is constructed, all peak points of the monitoring data change curve are obtained, and for any peak point, the value of the peak point is normalized to obtain a corresponding first normalized value, and the absolute value of the slope difference between the peak point and its left and right adjacent coordinate points is obtained, and the absolute value of the slope difference is normalized to obtain a corresponding second normalized value;

[0043] Obtaining a first time interval between the peak point and its left adjacent peak point, and a second time interval between the peak point and its right adjacent peak point, obtaining the inverse of a sum of the first time interval and the second time interval, and obtaining a product of the first normalized value, the second normalized value, and the inverse of the sum as a fluctuation characteristic value of the peak point;

[0044] The accumulated value of the fluctuation characteristic values ​​of all peak points of the monitoring data change curve is obtained as the data fluctuation index under the dimension, and the accumulated value of the data fluctuation index of each dimension of the N-dimensional monitoring data collected within the preset time period is used as the data fluctuation degree of the ultrasonic water meter.

[0045] In one implementation, taking the pth ultrasonic water meter as an example, for the jth dimension of the pth ultrasonic water meter, a corresponding monitoring data change curve is constructed for all monitoring data of the dimension, and the AMPD algorithm is used to obtain all peak points on the monitoring data change curve, wherein the AMPD algorithm belongs to the prior art and is not repeated here. Similarly, all peak points on the monitoring data change curve under each dimension of the pth ultrasonic water meter are obtained, and then according to the intervals of all peak points on the monitoring data change curve under each dimension of the pth ultrasonic water meter, the calculation expression for the data fluctuation degree of the pth ultrasonic water meter is obtained as follows:

[0046]

[0047] in, represents the data fluctuation degree of the pth ultrasonic water meter, N represents the number of dimensions, represents the number of peak points in the jth dimension, Represents the time interval (abscissa interval) between the i-th peak point and its left adjacent peak point on the j-th dimension, represents the time interval (abscissa interval) between the i-th peak point and its right adjacent peak point in the j-th dimension, norm() represents the normalization function, Represents the value (ordinate value) of the i-th peak point on the j-th dimension, Represents the absolute value of the difference between the slope value between the i-th peak point and its left adjacent coordinate point and the slope value between the i-th peak point and its right adjacent coordinate point in the j-th dimension.

[0048] It should be noted that the more peak points there are in each dimension, the more data points there are that fluctuate in the monitoring data collected under this dimension, and the higher the corresponding data fluctuation degree; for each peak point, The larger the value is, the more isolated the peak point is, and it does not form a continuous fluctuation, and the lower its contribution to the overall fluctuation degree is. It is the vertical axis span from the i-th peak point to the lowest minimum point on both sides. The larger the value of the peak point, the more violent the fluctuation of the peak point, the more difficult it is to have similar values, and the greater the contribution to the overall fluctuation characteristics; The larger the value of , the sharper the change at the peak point, the more difficult it is to have similar values, and the greater the contribution to the overall fluctuation characteristic degree; therefore, the fluctuation characteristic degrees of the p-th ultrasonic water meter in N dimensions are accumulated as the data fluctuation degree of the p-th ultrasonic water meter, that is, the data volatility characteristics in the two-dimensional matrix of the p-th ultrasonic water meter.

[0049] Similarly, the data fluctuation degree of each ultrasonic water meter can be obtained.

[0050] Step S102, obtain the data fluctuation degree of each ultrasonic water meter, and deeply stack the two-dimensional matrices of all ultrasonic water meters in order of all data fluctuation degrees from small to large to obtain a stacked matrix, and divide the stacked matrix into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter.

[0051] Step S101 can be used to obtain the data fluctuation degree and two-dimensional matrix of each ultrasonic water meter, and then the two-dimensional matrices of all ultrasonic water meters are deeply stacked in order of all data fluctuation degrees from small to large to obtain a stacked matrix. At this time, the stacked matrix is ​​in the form of a multidimensional space, and the stacked matrix is ​​divided into a gradual process from small data volatility to large data volatility in the vertical direction. Since the two-dimensional matrix area with small data volatility has greater adjacent similarity after adjacent aggregation, the stacked matrix is ​​divided into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter to avoid the situation where the adjacent similarity of the data is low due to the interlaced superposition of data with small volatility and data with large volatility.

[0052] Preferably, according to the data fluctuation degree of each ultrasonic water meter, the stacking matrix is ​​divided into two sub-stacking matrices, including:

[0053] Normalizing the data fluctuation degree of each ultrasonic water meter respectively to obtain a corresponding normalized value, and obtaining a preset normalized threshold value. If the normalized value corresponding to any ultrasonic water meter is less than or equal to the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with small data fluctuation. On the contrary, if the normalized value corresponding to any ultrasonic water meter is greater than the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with large data fluctuation.

[0054] The two-dimensional matrices with small data fluctuations in the stacked matrix are composed into a first sub-stacked matrix, and the two-dimensional matrices with large data fluctuations in the stacked matrix are composed into a second sub-stacked matrix.

[0055] In one embodiment, the normalization function norm is used to normalize the data fluctuation degree of each ultrasonic water meter to obtain a corresponding normalized value. The normalized value of the pth ultrasonic water meter is recorded as , set the normalization threshold to 0.3, there is no limit here, and it can be adjusted according to the scene requirements. Then, according to the normalization threshold, the stacking matrix is ​​divided into two parts. One part is the first sub-stacked matrix composed of two-dimensional matrices with normalization values ​​less than or equal to 0.3, that is, the part with small data fluctuations in the stacking matrix, and the other part is the second sub-stacked matrix composed of two-dimensional matrices with normalization values ​​greater than 0.3, that is, the part with large data fluctuations in the stacking matrix. In short, according to the normalization threshold of 0.3, the stacking matrix is ​​divided into two parts in the vertical direction, upper and lower.

[0056] At this point, the stacking matrix is ​​divided into a first sub-stacked matrix and a second sub-stacked matrix.

[0057] Step S103, normalizing each element in each sub-stacked matrix to obtain a corresponding normalized matrix, and obtaining an adaptive compression path for each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility.

[0058] In order to avoid the influence of data dimension, each element in each sub-stacked matrix is ​​normalized to obtain the corresponding normalized matrix. Since the normalized matrix corresponding to the first sub-stacked matrix belongs to the part with small data fluctuation, the adjacent similarity of its data is relatively high. Therefore, the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix can be obtained according to the neighborhood similarity of each data in the normalized matrix corresponding to the first sub-stacked matrix. The specific acquisition method is:

[0059] For the normalized matrix corresponding to the first sub-stacked matrix, construct the three-dimensional space of the normalized matrix to obtain three spatial planes, for any spatial plane, obtain the absolute value of the data difference between each data in the spatial plane and each neighboring data within its preset neighborhood range, obtain the accumulated value of the absolute value of the data difference of each data in the spatial plane, use an exponential function with a natural constant as the base to inversely normalize the sum of the accumulated values ​​of the absolute values ​​of the data differences of all data, and use the result obtained as the overall similarity eigenvalue of the spatial plane;

[0060] According to the overall similarity eigenvalue of each of the spatial planes, the spatial plane corresponding to the largest overall similarity eigenvalue is obtained as the traversal plane, and a horizontal Z-shaped traversal method is used on the traversal plane as the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix.

[0061] In one embodiment, since the normalized matrix corresponding to the first sub-stacked matrix is ​​a three-dimensional two-dimensional matrix, a three-dimensional space of the normalized matrix is ​​constructed, wherein the three-dimensional space refers to the horizontal and column of the last two-dimensional matrix in the normalized matrix as the x-direction and the y-direction, and the vertical direction (that is, the depth direction of the normalized matrix) as the z-direction, and the corresponding three spatial planes are plane xy, plane xz, and plane yz. For plane xy, the neighborhood data of each data in the normalized matrix in the eight neighborhoods on plane xy are obtained respectively, and then the overall similarity eigenvalues ​​of all data of the normalized matrix on plane xy are obtained according to the numerical difference between each data and its neighborhood data, and the calculation expression of the overall similarity eigenvalue is:

[0062]

[0063] in, represents the global similarity eigenvalue of the qth spatial plane of the normalized matrix, Represents the value of the i-th data in the normalized matrix, Represents the value of the jth data in the eight-neighborhood of the i-th data of the normalized matrix, represents the amount of data in the eight-neighborhood of the i-th data of the normalized matrix on the q-th spatial plane, w represents the number of data in the normalized matrix, || represents the absolute value sign, and exp() represents the exponential function with a natural constant as the base.

[0064] It should be noted that The smaller the value is, the smaller the difference between the corresponding two data is, and the higher the similarity is. By accumulating the numerical differences between all the data in the normalized matrix and each data in its eight neighborhoods, the smaller the accumulated value is, the smaller the difference between the data in the normalized matrix is, the greater the overall similarity of all the data in the normalized matrix on the qth spatial plane is, and the greater the overall similarity eigenvalue of the qth spatial plane is.

[0065] Similarly, according to the numerical difference between each data in the normalized matrix and each data in the eight neighborhoods in different spatial planes, the overall similarity eigenvalues ​​of all the data in the normalized matrix on each spatial plane can be obtained respectively, and then the spatial plane corresponding to the largest overall similarity eigenvalue is used as the traversal plane of the normalized matrix, and the data of the normalized matrix is ​​traversed according to the traversal plane and in a horizontal Z-shape on the traversal plane. This traversal method obtains the strongest adjacent similarity between the traversed data, and the greater the repeatability of the differential data obtained by subsequent differentiation based on the traversed data. Therefore, the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix is ​​a horizontal Z-shaped traversal method on the traversal plane.

[0066] Furthermore, for the normalized matrix corresponding to the second sub-stacked matrix, which is the part with large data fluctuations, the adjacent similarities on all planes are low. Therefore, the multidimensional correlation of the ultrasonic water meter collected data is combined to evaluate the distribution of data with large fluctuations.

[0067] The water in the pipe will produce a thermal expansion effect when heated. When the water temperature rises, the volume of the water will expand. The pipes monitored by the water meter are usually closed, and the volume expansion will cause the water pressure to increase; conversely, when the water temperature drops, the volume of the water shrinks and the water pressure will decrease. On the other hand, water pressure will also affect the flow rate of water. High pressure usually leads to a higher flow rate, which will also affect the heat transfer and heat dissipation of water in the pipe. High-speed flowing water will reduce heat loss and maintain a higher water temperature; low-speed flowing water is easier to dissipate heat and the temperature will drop. Therefore, there is a certain degree of correlation between the multi-dimensional monitoring data collected by a single ultrasonic water meter. When the inside of the water pipe monitored by the ultrasonic water meter is narrow, the internal space is small, and the water temperature, water pressure, and flow rate are more likely to be correlated. The higher the fluctuation similarity of the data in the three dimensions of instantaneous flow, water temperature and water pressure, the stronger the correlation of the corresponding multidimensional data sequence. The more ultrasonic water meters with strong correlation, the more monitoring scenes of this type are contained in the ultrasonic water meters connected to the Internet of Things. The longitudinal Z-shaped data traversal of the spatial plane xy can improve the adjacent similarity of the data with large fluctuations. Therefore, for the normalized matrix corresponding to the second sub-stacked matrix, the adaptive compression path of the normalized matrix is ​​obtained according to the neighborhood similarity of each data in the normalized matrix and the overall data volatility. The specific acquisition method is:

[0068] For any ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, the Pearson correlation coefficient between the normalized data sequences of every two dimensions is calculated according to the normalized data sequence of each dimension respectively corresponding to the N-dimensional monitoring data of the ultrasonic water meter in the normalized matrix, and the difference between each Pearson correlation coefficient and a constant 1 is obtained as the multi-dimensional correlation eigenvalue of the ultrasonic water meter by adding a difference cumulative value;

[0069] According to the multi-dimensional correlation eigenvalues ​​of each ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, a cumulative value of the multi-dimensional correlation eigenvalues ​​is obtained, and the cumulative value of the multi-dimensional correlation eigenvalues ​​is negatively mapped by using an exponential function with a natural constant as the base, and the obtained mapping value is used as the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix;

[0070] According to the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix, an adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained.

[0071] In one embodiment, the calculation expression of the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is:

[0072]

[0073] Wherein, W represents the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix, exp() represents an exponential function with a natural constant as the base, N represents the number of dimensions of the monitoring data, and m represents the number of ultrasonic water meters corresponding to the normalized matrix corresponding to the second sub-stacked matrix. represents the normalized data sequence of the ath dimension of the pth ultrasonic water meter, represents the normalized data sequence of the bth dimension of the pth ultrasonic water meter, Represents the Pearson correlation coefficient between the normalized data series of the two dimensions of the pth ultrasonic water meter, and 1 represents a constant.

[0074] It should be noted that for any ultrasonic water meter in the normalized matrix, the correlation between the data sequences of each two dimensions of the ultrasonic water meter is used to evaluate whether the two dimensions are associated, that is, by obtaining the Pearson correlation coefficient. The closer the Pearson correlation coefficient is to 1, the stronger the positive correlation between the data sequences of the two dimensions. Since the Pearson correlation coefficient has negative values, it is used The result is quantified as a positive value, which means that the smaller the result, the stronger the positive correlation of the data series in the two dimensions. The positive correlation of the data series between the dimensions of all ultrasonic water meters in the normalized matrix is ​​accumulated, and the accumulated result is used as the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix.

[0075] After obtaining the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix, the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained according to the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix. The specific acquisition method is:

[0076] Obtaining a preset overall data association index threshold, if the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​greater than or equal to the overall data association index threshold, using a longitudinal Z-shaped traversal method on a preset traversal plane as an adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix;

[0077] If the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​less than the overall data association index threshold, the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained, and a horizontal Z-shaped traversal method is used on the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix as the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix.

[0078] In one embodiment, the threshold of the overall data association index is set to 0.6, which can be adaptively adjusted according to the scene. If the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​greater than or equal to 0.6, it means that the correlation between the multidimensional data in the normalized matrix corresponding to the second sub-stacked matrix is ​​relatively strong, and the data in the normalized matrix is ​​traversed in a longitudinal Z-shaped manner on the spatial plane xy, that is, the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix is ​​a traversal method in a longitudinal Z-shaped manner on the spatial plane xy. On the contrary, if the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​less than 0.6, then according to the method for obtaining the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix, the corresponding traversal plane is obtained by analyzing the overall similarity eigenvalues ​​of each spatial plane in the normalized matrix corresponding to the second sub-stacked matrix, and then the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix is ​​traversed in a transverse Z-shaped manner as the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix.

[0079] So far, the adaptive compression path of each normalized matrix is ​​obtained.

[0080] Step S104, obtaining a data sequence to be compressed according to the adaptive compression path of each normalized matrix, and performing run-length encoding on the data sequence to be compressed to obtain corresponding compressed data.

[0081] After obtaining the adaptive path compression path of each normalized matrix, for any normalized matrix, according to the adaptive compression path of the normalized matrix, obtain the data sequence to be compressed, including: for any normalized matrix, traverse the data in the normalized matrix in sequence according to the adaptive compression path of the normalized matrix to obtain a traversed data sequence, perform first-order difference on the traversed data sequence to obtain a difference sequence, and place the difference sequence after the first data in the traversed data sequence to form a data sequence to be compressed.

[0082] In one embodiment, if the adaptive compression path of the normalized matrix adopts a horizontal Z-shaped traversal method of the traversal plane, the specific traversal process is: according to the traversal plane, the traversal starts from the upper left corner of the corresponding uppermost traversal plane in the normalized matrix, and the plane is traversed in a horizontal Z-shaped manner until the traversal of the plane is completed, and the traversal of the next plane is continued, thereby iterating until all the data in the normalized matrix are traversed, and the data are stored in the traversal data sequence according to the traversal order. If the adaptive compression path of the normalized matrix adopts a vertical Z-shaped traversal method of the spatial plane xy, the specific traversal process is: starting from the upper left corner of the uppermost layer corresponding to the plane xy of the normalized matrix, and the plane is traversed in a vertical Z-shaped manner until the traversal of the plane is completed, and the traversal of the next plane is continued, thereby iterating until all the data in the normalized matrix are traversed, and the data are stored in the traversal data sequence according to the traversal order. At this point, the traversed data sequence obtained by traversing the data of each normalized matrix can be obtained. At this time, the adjacent similarity of the data between the traversed data sequences obtained is high. Then, for each traversed data sequence, the repeatability of the data is improved by performing differential processing on the traversed data sequence. Specifically, the traversed data sequence is subjected to a first-order difference to obtain a differential sequence, and the differential sequence is placed after the first data in the traversed data sequence to form a data sequence to be compressed. It should be noted that the data sequence to be compressed also corresponds to a sign sequence. For a negative differential value in the sequence to be compressed, the sign of the differential value is set to 0. If it is not a negative number, the sign of the differential value is set to 1.

[0083] Furthermore, after obtaining the data sequence to be compressed, the RLE compression algorithm is used to compress the data sequence to obtain compressed data. At this time, the purpose of compressing the monitoring data of all ultrasonic water meters is achieved. The data sequence to be compressed is obtained by improving the sequence difference of adjacent similarities, which improves the repetition degree of adjacent data and improves the compression efficiency of the RLE compression algorithm for multi-dimensional data collected by the Internet of Things ultrasonic water meters.

[0084] In summary, the embodiment of the present invention is for any ultrasonic water meter, obtains N-dimensional monitoring data collected by the ultrasonic water meter within a preset time period, and constructs the N-dimensional monitoring data collected within the preset time period into an N*M two-dimensional matrix, and obtains the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset time period; obtains the data fluctuation degree of each ultrasonic water meter, and deeply stacks the two-dimensional matrices of all ultrasonic water meters in the order of all data fluctuation degrees from small to large to obtain a stacked matrix, and divides the stacked matrix into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter; normalizes each element in each of the sub-stacked matrices to obtain a corresponding normalized matrix, and obtains the adaptive compression path of each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility; obtains a data sequence to be compressed according to the adaptive compression path of each normalized matrix, and runs the data sequence to be compressed to obtain corresponding compressed data. Among them, by performing data fluctuation analysis on the N-dimensional monitoring data of multiple ultrasonic water meters connected to the Internet of Things, the two-dimensional matrices of all ultrasonic water meters are deeply stacked to obtain a stacked matrix, that is, a multi-dimensional space is constructed, and then based on the multi-dimensional space where the stacked matrix is ​​located, the stacked matrix is ​​divided into two sub-stacked matrices with large data volatility and small data volatility, so as to be able to subsequently obtain adjacent compression paths with high similarity in the form of differences, so that the possibility of the data after differentiation being repeated data is higher. Therefore, data similarity or overall data volatility analysis is performed on each sub-stacked matrix to obtain an adaptive compression path for each sub-stacked matrix, so that the repeatability in the data sequence to be compressed obtained based on the adaptive compression path is higher, thereby improving the compression efficiency of run-length encoding of the data sequence to be compressed.

[0085] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A data compression and storage method for an Internet of Things ultrasonic water meter, characterized in that: The method comprises: For any ultrasonic water meter, obtain N-dimensional monitoring data collected by the ultrasonic water meter within a preset period of time, and construct the N-dimensional monitoring data collected within the preset period of time into an N*M two-dimensional matrix, and obtain the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset period of time; Obtaining the data fluctuation degree of each ultrasonic water meter, deeply stacking the two-dimensional matrices of all ultrasonic water meters in the order of all data fluctuation degrees from small to large to obtain a stacked matrix, and dividing the stacked matrix into two sub-stacked matrices according to the data fluctuation degree of each ultrasonic water meter; Normalizing each element in each of the sub-stacked matrices to obtain a corresponding normalized matrix, and obtaining an adaptive compression path for each of the normalized matrices according to the neighborhood similarity of each data in each of the normalized matrices and the overall data volatility; Acquire a data sequence to be compressed according to the adaptive compression path of each normalized matrix, and perform run-length encoding on the data sequence to be compressed to obtain corresponding compressed data; The stacking matrix is ​​divided into two sub-stacking matrices according to the data fluctuation degree of each ultrasonic water meter, including: Normalizing the data fluctuation degree of each ultrasonic water meter respectively to obtain a corresponding normalized value, and obtaining a preset normalized threshold value. If the normalized value corresponding to any ultrasonic water meter is less than or equal to the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with small data fluctuation. On the contrary, if the normalized value corresponding to any ultrasonic water meter is greater than the normalized threshold value, the two-dimensional matrix of the ultrasonic water meter in the stacked matrix is ​​regarded as a two-dimensional matrix with large data fluctuation. The two-dimensional matrices with small data fluctuations in the stacked matrix are formed into a first sub-stacked matrix, and the two-dimensional matrices with large data fluctuations in the stacked matrix are formed into a second sub-stacked matrix; The step of obtaining the adaptive compression path of each normalized matrix according to the neighborhood similarity of each data in each normalized matrix and the overall data volatility includes: For the normalized matrix corresponding to the first sub-stacked matrix, construct the three-dimensional space of the normalized matrix to obtain three spatial planes, for any spatial plane, obtain the absolute value of the data difference between each data in the spatial plane and each neighboring data within its preset neighborhood range, obtain the accumulated value of the absolute value of the data difference of each data in the spatial plane, use an exponential function with a natural constant as the base to inversely normalize the sum of the accumulated values ​​of the absolute values ​​of the data differences of all data, and use the result obtained as the overall similarity eigenvalue of the spatial plane; According to the overall similarity eigenvalue of each of the spatial planes, the spatial plane corresponding to the largest overall similarity eigenvalue is obtained as the traversal plane, and a horizontal Z-shaped traversal manner is used on the traversal plane as the adaptive compression path of the normalized matrix corresponding to the first sub-stacked matrix; For any ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, the Pearson correlation coefficient between the normalized data sequences of every two dimensions is calculated according to the normalized data sequence of each dimension respectively corresponding to the N-dimensional monitoring data of the ultrasonic water meter in the normalized matrix, and the difference between each Pearson correlation coefficient and a constant 1 is obtained as the multi-dimensional correlation eigenvalue of the ultrasonic water meter by adding a difference cumulative value; According to the multi-dimensional correlation eigenvalues ​​of each ultrasonic water meter in the normalized matrix corresponding to the second sub-stacked matrix, a cumulative value of the multi-dimensional correlation eigenvalues ​​is obtained, and the cumulative value of the multi-dimensional correlation eigenvalues ​​is negatively mapped by using an exponential function with a natural constant as the base, and the obtained mapping value is used as the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix; Obtaining a preset overall data association index threshold, if the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​greater than or equal to the overall data association index threshold, using a longitudinal Z-shaped traversal method on a preset traversal plane as an adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix; If the overall data association index of the normalized matrix corresponding to the second sub-stacked matrix is ​​less than the overall data association index threshold, the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix is ​​obtained, and a horizontal Z-shaped traversal method is used on the traversal plane of the normalized matrix corresponding to the second sub-stacked matrix as the adaptive compression path of the normalized matrix corresponding to the second sub-stacked matrix.

2. The data compression and storage method of an Internet of Things ultrasonic water meter according to claim 1 is characterized in that: The step of obtaining the data fluctuation degree of the ultrasonic water meter according to the data change of the N-dimensional monitoring data collected within the preset time period includes: For any dimension of the N-dimensional monitoring data collected within the preset time period, a monitoring data change curve under the dimension is constructed, all peak points of the monitoring data change curve are obtained, and for any peak point, the value of the peak point is normalized to obtain a corresponding first normalized value, and the absolute value of the slope difference between the peak point and its left and right adjacent coordinate points is obtained, and the absolute value of the slope difference is normalized to obtain a corresponding second normalized value; Obtaining a first time interval between the peak point and its left adjacent peak point, and a second time interval between the peak point and its right adjacent peak point, obtaining the inverse of a sum of the first time interval and the second time interval, and obtaining a product of the first normalized value, the second normalized value, and the inverse of the sum as a fluctuation characteristic value of the peak point; The accumulated value of the fluctuation characteristic values ​​of all peak points of the monitoring data change curve is obtained as the data fluctuation index under the dimension, and the accumulated value of the data fluctuation index of each dimension of the N-dimensional monitoring data collected within the preset time period is used as the data fluctuation degree of the ultrasonic water meter.

3. The data compression and storage method of an Internet of Things ultrasonic water meter according to claim 1 is characterized in that: The step of obtaining a data sequence to be compressed according to the adaptive compression path of each normalized matrix comprises: For any normalized matrix, the data in the normalized matrix are traversed in sequence according to the adaptive compression path of the normalized matrix to obtain a traversed data sequence, and the traversed data sequence is subjected to a first-order difference to obtain a differential sequence, and the differential sequence is placed after the first data in the traversed data sequence to form a data sequence to be compressed.

Images