Data migration optimization transmission method
By evaluating the importance of water level data and dynamically updating the dictionary table of the LZW algorithm, the problem of loss or damage of important data in water level data transmission in the prior art is solved, and the security and accuracy of data transmission are achieved.
Patent Information
- Application Number
- CN202411192895.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-03
AI Technical Summary
Existing water level data compression methods fail to effectively consider the importance of data, resulting in the possibility of loss or damage during transmission, affecting the security and accuracy of the data.
By obtaining the water level data, precipitation and water flow of the target monitoring point, calculate the stationarity, rainfall rate and desirability of each water level data, and combine the data from the upstream monitoring point to evaluate the single importance and contrast importance of each water level data. , dynamically update the dictionary table of the LZW algorithm to enable important data encoding to ensure transmission security.
Improve the efficiency and accuracy of data transmission, ensure the safety of important data during transmission, and ensure priority transmission of key data for hydrological prediction and water resource management.
Smart Images

Figure CN119071670A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital information transmission, and in particular to a data migration optimization transmission method. Background Art
[0002] Water level data records are very important in many fields such as hydrology, water resources management, flood prevention and disaster reduction. The compression of large amounts of water level data can reduce the use of storage space, which is especially important for large-scale hydrological data. By compressing data, storage costs can be saved, the efficiency of data transmission can be improved, and the possibility of errors in the transmission process can be reduced, which speeds up data processing and enables hydrological researchers and water resources managers to obtain useful information more quickly and make corresponding decisions.
[0003] Generally, when the LZW algorithm is used to compress water level data, the initial dictionary is expanded according to the order in which the combined data appear. If the combined data is in front, the code is placed in front and compressed. However, this method does not take the importance of the data into consideration. Suppose there are two combined data A and B. If A appears in front, its code is short and the risk of loss during transportation is lower. However, A is not necessarily more important than B. Suppose A is unimportant data when the water level does not change, and the water level changes greatly at B. At this time, B data should be recorded as it is likely to be flooded, making its code closer to the front. This may increase the risk of important data being lost or damaged during transmission, making it difficult to ensure the security of important data transmission. Summary of the invention
[0004] The present invention provides a data migration optimization transmission method to solve the existing problems.
[0005] A data migration optimization transmission method of the present invention adopts the following technical solution:
[0006] An embodiment of the present invention provides a data migration optimization transmission method, the method comprising the following steps:
[0007] Obtain the water level data of the target monitoring point at each moment, the precipitation and water flow at each water level data moment, and the water level data at the same moment last year at each water level data moment; obtain the water level data of all upstream monitoring points at each moment and the precipitation at each water level data moment; obtain the distance from the target monitoring point to each upstream monitoring point;
[0008] According to all the water level data of the target monitoring point, the stability of each water level data of the target monitoring point is obtained; according to the precipitation at the time when all the water level data of the target monitoring point are located, the rainfall rate at the time when each water level data of the target monitoring point is located is obtained; according to the stability of each water level data of the target monitoring point, the rainfall rate at the time, the precipitation at the time when all the water level data are located, and the water flow, the desirability of each water level data of the target monitoring point is obtained;
[0009] According to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year when each water level data of the target monitoring point is located, the single importance of each water level data of the target monitoring point is obtained; according to the single importance of all water level data of the target monitoring point, the comparative importance of each water level data of the target monitoring point is obtained;
[0010] Each water level data of the target monitoring point is used as each data of the initial dictionary; according to the comparative importance of all the data of the initial dictionary and all the water level data of the target monitoring point, the coding position front rate of each data is obtained; according to the coding position front rate of all the data, data transmission is performed.
[0011] Furthermore, the step of obtaining the stability of each water level data of the target monitoring point based on all water level data of the target monitoring point includes the following specific steps:
[0012] The a most recent water level data of each water level data of the target monitoring point in time are used as the comparison water level data of each water level data, where a is the preset number of comparison data;
[0013] According to each water level data of the target monitoring point and all the comparison water level data of each water level data, the stationarity of each water level data of the target monitoring point is obtained.
[0014] Furthermore, the specific calculation formula for obtaining the stability of each water level data of the target monitoring point based on each water level data of the target monitoring point and all the comparison water level data of each water level data is:
[0015]
[0016] Where P' i Indicates the stability of the i-th water level data of the target monitoring point; a i is the number of comparison water level data of the i-th water level data of the target monitoring point; w i,j represents the jth comparison water level data of the i-th water level data of the target monitoring point; μ iRepresents the average value of all comparison water level data of the i-th water level data of the target monitoring point; exp() is an exponential function with a natural constant as the base.
[0017] Furthermore, the step of obtaining the rainfall rate at the moment of each water level data of the target monitoring point according to the precipitation at the moment of all water level data of the target monitoring point includes the following specific steps:
[0018] The normalized value of the average precipitation at the time when all the comparison water level data of each water level data of the target monitoring point are located is recorded as the rainfall rate at the time when each water level data of the target monitoring point is located.
[0019] Furthermore, the desirability of each water level data of the target monitoring point is obtained according to the stability of each water level data of the target monitoring point, the rainfall rate at the time, the precipitation and water flow at the time of all water level data, and the specific steps include the following:
[0020] The water level data of the target monitoring point is used as the horizontal axis and the precipitation as the vertical axis to construct a two-dimensional coordinate system of precipitation;
[0021] In the two-dimensional precipitation coordinate system, the least square method is used to perform a straight line fitting on the data points corresponding to each water level data of the target monitoring point and all its comparison water level data, and the fitting precipitation corresponding to each water level data of the target monitoring point is obtained;
[0022] The water level data of the target monitoring point is used as the horizontal axis and the water flow rate is used as the vertical axis to construct a two-dimensional coordinate system of water flow rate;
[0023] In the two-dimensional coordinate system of water flow, the least square method is used to perform straight line fitting on the data points corresponding to each water level data of the target monitoring point and all the comparative water level data, so as to obtain the fitted water flow corresponding to each water level data of the target monitoring point;
[0024] According to the stationarity of each water level data of the target monitoring point, the rainfall rate at the time, the corresponding fitted precipitation and the fitted water flow, the desirability of each water level data of the target monitoring point is obtained.
[0025] Furthermore, the specific calculation formula for obtaining the desirability of each water level data of the target monitoring point according to the stability of each water level data of the target monitoring point, the rainfall rate at the time, the corresponding fitted precipitation and the fitted water flow is:
[0026]
[0027] In the formula, P i Indicates the desirability of the i-th water level data of the target monitoring point; L irepresents the rainfall rate at the moment when the i-th water level data of the target monitoring point is located; b represents the preset rainfall rate threshold; P' i represents the stability of the i-th water level data of the target monitoring point; v i Indicates the precipitation at the time of the i-th water level data of the target monitoring point; v' i represents the fitted precipitation corresponding to the i-th water level data of the target monitoring point; r i Indicates the water flow at the time when the i-th water level data of the target monitoring point is located; r' i The fitted water flow corresponding to the i-th water level data of the target monitoring point; || is the absolute value function; exp[] is the exponential function with the natural constant as the base.
[0028] Furthermore, according to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year when each water level data of the target monitoring point is located, the specific calculation formula for obtaining the single importance of each water level data of the target monitoring point is:
[0029]
[0030] Where K' i Indicates the single importance of the i-th water level data of the target monitoring point; P" i represents the desirability of the i-th water level data of the target monitoring point; m represents the number of upstream monitoring points; D h represents the distance from the hth upstream monitoring point to the target monitoring point; v h,i+1 represents the precipitation at the time of the i+1th water level data of the hth upstream monitoring point; v h,i represents the precipitation at the time of the i-th water level data of the h-th upstream monitoring point; v h,i-1 represents the precipitation at the time of the i-1th water level data of the hth upstream monitoring point; w i represents the i-th water level data of the target monitoring point; w' i It represents the water level data of the i-th water level data of the target monitoring point at the same time last year; || is the absolute value function; exp[] is the exponential function with natural constant as the base.
[0031] Furthermore, the specific calculation formula for obtaining the comparative importance of each water level data of the target monitoring point according to the single importance of all water level data of the target monitoring point is:
[0032] K” i =norm(K' i+1 +K' i-1 )×K' i
[0033] In the formula, K” i Indicates the comparative importance of the i-th water level data of the target monitoring point; K' i+1 Indicates the single importance of the i+1th water level data of the target monitoring point; K' i-1 Indicates the single importance of the i-1th water level data of the target monitoring point; K' i It represents the single importance of the i-th water level data of the target monitoring point; norm() is a linear normalization function.
[0034] Furthermore, the method of obtaining the coding position front rate of each data according to the comparison importance of all data in the initial dictionary and all water level data of the target monitoring point includes the following specific steps:
[0035] In the initial dictionary, the same data is recorded as a data class;
[0036] According to the number of data in all data classes of the initial dictionary and the comparative importance of all water level data of the target monitoring point, the coding position front rate of each data is obtained.
[0037] Furthermore, the method of obtaining the coding position front rate of each data according to the number of data in all data classes of the initial dictionary and the comparative importance of all water level data of the target monitoring point includes the following specific steps:
[0038] The product of the number of data in the data class where each data in the initial dictionary is located and the sum of the comparative importance of the water level data of the target monitoring point corresponding to all the data in the data class where each data in the initial dictionary is located is recorded as the encoding position front rate of each data.
[0039] The beneficial effects of the technical solution of the present invention are:
[0040] In the embodiment of the present invention, the water level data of the target monitoring point at each moment, the precipitation and water flow at each water level data moment, and the water level data at the same moment last year when each water level data moment is located are obtained; the water level data of all upstream monitoring points at each moment and the precipitation at each water level data moment are obtained; the distance from the target monitoring point to each upstream monitoring point is obtained to provide a data basis for subsequent data analysis and calculation; based on all the water level data of the target monitoring point, the stability of each water level data of the target monitoring point is obtained, which is helpful to determine whether the water level data presents a stable trend; based on the precipitation at the moment when all the water level data of the target monitoring point are located, the rainfall rate at the moment when each water level data of the target monitoring point is located is obtained, which is convenient for analyzing the desirability of each water level data; based on the stability of each water level data of the target monitoring point, the rainfall rate at the moment, the precipitation and water flow at the moment when all the water level data are located, the number of each water level data of the target monitoring point is obtained. According to the desirability of each water level data, the credibility and validity of each water level data are determined; according to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year when each water level data of the target monitoring point is located, the single importance of each water level data of the target monitoring point is obtained; according to the single importance of all water level data of the target monitoring point, the comparative importance of each water level data of the target monitoring point is obtained, and the most critical data for hydrological prediction and water resources management can be transmitted preferentially; each water level data of the target monitoring point is used as each data of the initial dictionary; according to the comparative importance of all data of the initial dictionary and all water level data of the target monitoring point, the coding position front rate of each data is obtained, which is helpful to improve the efficiency and accuracy of data transmission; according to the coding position front rate of all data, data transmission is performed. So far, the present invention dynamically updates the dictionary table of the LZW algorithm based on the importance of the data, so that the important data is more forward in the coding table, ensuring the safety during transportation. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 The present invention is a flowchart of the steps of a data migration optimization transmission method. DETAILED DESCRIPTION
[0043] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the data migration optimization transmission method proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0045] The following is a detailed description of a data migration optimization transmission method provided by the present invention with reference to the accompanying drawings.
[0046] See also Figure 1 , which shows a flow chart of a data migration optimization transmission method provided by an embodiment of the present invention, the method comprising the following steps:
[0047] Step S001: Obtain the water level data of the target monitoring point at each moment, the precipitation and water flow at each moment of water level data, and the water level data at the same moment of last year when each water level data is located; obtain the water level data of all upstream monitoring points at each moment and the precipitation at each moment of water level data; obtain the distance from the target monitoring point to each upstream monitoring point.
[0048] Select a target monitoring point and several upstream monitoring points in the river; collect water level data of the target monitoring point at each moment, precipitation and water flow at each moment of water level data, and water level data at the same moment of each moment of water level data last year;
[0049] Collect the water level data of all upstream monitoring points at each moment and the precipitation at each moment of water level data; collect the distance from the target monitoring point to each upstream monitoring point;
[0050] It should be noted that the preset time interval of this embodiment is half an hour, which is used as an example for description. Other time intervals can be set in other implementation modes, and this embodiment does not limit it. The water level data of the river is collected every half hour using the buoy method, the precipitation is collected using a rain gauge, and the water flow data is collected using a flow velocity instrument.
[0051] Step S002: Based on all water level data of the target monitoring point, obtain the stability of each water level data of the target monitoring point; based on the precipitation at the moment when all water level data of the target monitoring point are located, obtain the rainfall rate at the moment when each water level data of the target monitoring point is located; based on the stability of each water level data of the target monitoring point, the rainfall rate at the moment, the precipitation at the moment when all water level data are located, and the water flow, obtain the desirability of each water level data of the target monitoring point.
[0052] For river water level data, the water level data will not change greatly in a short period of time, but the buoy may be affected by external forces such as wind and waves, which makes the measurement of water level data more volatile, so its stability characteristics are first determined based on volatility.
[0053] The preset comparison data number a=10 in this embodiment is described as an example. Other values may be set in other implementations and are not limited in this embodiment.
[0054] The a most recent water level data of each water level data of the target monitoring point in time is used as the comparison water level data of each water level data;
[0055] According to each water level data of the target monitoring point and all the comparison water level data of each water level data, the corresponding calculation formula for the stability of each water level data of the target monitoring point is obtained as follows:
[0056]
[0057] Where P' i Indicates the stability of the i-th water level data of the target monitoring point; a i is the number of comparison water level data of the i-th water level data of the target monitoring point; w i,j represents the jth comparison water level data of the i-th water level data of the target monitoring point; μ i Represents the average value of all comparison water level data of the i-th water level data of the target monitoring point; exp() is an exponential function with a natural constant as the base. This embodiment uses exp(-) to present the inverse proportional relationship and normalization processing. The implementer can set the inverse proportional function and normalization function according to the actual situation.
[0058] It should be noted that the variance of the i-th water level data of the target monitoring point and its comparison data is used. It represents the volatility of the i-th water level data of the target monitoring point. The smaller the volatility, the higher the stability of the i-th water level data of the target monitoring point.
[0059] If data is collected on a river during the rainfall period, rainfall has the greatest impact on water level data. The more rainfall there is, the higher the water level will be. Therefore, the water level data is linearly proportional to the precipitation. If it is not during the rainfall period, water flow has the greatest impact on water level data. Therefore, water level data is linearly proportional to water flow. The credibility of each water level data is determined based on this.
[0060] According to the precipitation at the time when all the comparison water level data of each water level data of the target monitoring point are located, the corresponding calculation formula for the rainfall rate at the time when each water level data of the target monitoring point is located is:
[0061]
[0062] Where, L i Indicates the rainfall rate at the moment when the i-th water level data of the target monitoring point is located; a i is the number of comparison data of the i-th water level data of the target monitoring point; v i,j It represents the precipitation at the moment of the j-th comparison water level data of the ith water level data of the target monitoring point; norm() is a linear normalization function that normalizes the data value to the interval [0,1].
[0063] The water level data of the target monitoring point is used as the horizontal axis and the precipitation as the vertical axis to construct a two-dimensional coordinate system of precipitation;
[0064] In the two-dimensional coordinate system of precipitation, the least square method is used to perform a straight line fitting on the data points corresponding to each water level data of the target monitoring point and all its comparison water level data, so as to obtain the fitted precipitation corresponding to each water level data of the target monitoring point, wherein the least square method is a well-known technology;
[0065] The water level data of the target monitoring point is used as the horizontal axis and the water flow rate is used as the vertical axis to construct a two-dimensional coordinate system of water flow rate;
[0066] In the two-dimensional coordinate system of water flow, the least square method is used to perform straight line fitting on the data points corresponding to each water level data of the target monitoring point and all the comparative water level data, so as to obtain the fitted water flow corresponding to each water level data of the target monitoring point, wherein the least square method is a well-known technology;
[0067] The rainfall rate threshold b=0.1 is preset in this embodiment, and is described as an example. Other values may be set in other implementations, and this embodiment does not limit this.
[0068] According to the stability of each water level data of the target monitoring point, the rainfall rate at the time, the corresponding fitted precipitation and fitted water flow, the corresponding calculation formula for the desirability of each water level data of the target monitoring point is obtained as follows:
[0069]
[0070] In the formula, P i Indicates the desirability of the i-th water level data of the target monitoring point; L i represents the rainfall rate at the moment when the i-th water level data of the target monitoring point is located; b represents the preset rainfall rate threshold; P' i represents the stability of the i-th water level data of the target monitoring point; v i Indicates the precipitation at the time of the i-th water level data of the target monitoring point; v' i represents the fitted precipitation corresponding to the i-th water level data of the target monitoring point; r i Indicates the water flow at the time when the i-th water level data of the target monitoring point is located; r' i The fitted water flow corresponding to the i-th water level data of the target monitoring point; || is the absolute value function; exp[] is an exponential function with a natural constant as the base. This embodiment uses exp[-] to present the inverse proportional relationship and normalization processing. The implementer can set the inverse proportional function and normalization function according to the actual situation.
[0071] It should be noted that L i >b, it is considered that the i-th water level data of the target monitoring point is in the rainfall period, and the stability of the i-th water level data of the target monitoring point is P' i The larger the value, the higher the desirability of the i-th water level data of the target monitoring point; |v i -v' i | represents the absolute value of the difference between the precipitation at the time of the i-th water level data of the target monitoring point and its corresponding fitted precipitation. The smaller the value, the more consistent the linear positive proportional relationship between the i-th water level data of the target monitoring point and the precipitation at the time of the i-th water level data, that is, the higher the desirability of the i-th water level data of the target monitoring point; L i When ≤b, it is considered that the i-th water level data of the target monitoring point is not in the rainfall period, and the same can be said.
[0072] Step S003: Obtain the single importance of each water level data of the target monitoring point based on the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data of each water level data of the target monitoring point at the same time last year; obtain the comparative importance of each water level data of the target monitoring point based on the single importance of all water level data of the target monitoring point.
[0073] If the credibility of water level data is high, it should be recorded as a priority; if the difference with the data of previous years is small, the impact of rainfall on the importance of water level data will not be great; if the difference with the data of previous years is large, rainfall will have a great impact on it (the closer the distance to the target monitoring point, the greater the impact of rainfall on water flow data, and the greater the impact on it should be; the farther the distance from the monitoring point, the smaller the impact of rainfall on water flow data, and the smaller the impact on importance).
[0074] According to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year when each water level data of the target monitoring point is located, the corresponding calculation formula for the single importance of each water level data of the target monitoring point is obtained as follows:
[0075]
[0076] In the formula, K' i Indicates the single importance of the i-th water level data of the target monitoring point; P" i ' represents the desirability of the i-th water level data of the target monitoring point; m represents the number of upstream monitoring points; D h represents the distance from the hth upstream monitoring point to the target monitoring point; v h,i+1 represents the precipitation at the time of the i+1th water level data of the hth upstream monitoring point; v h,i represents the precipitation at the time of the i-th water level data of the h-th upstream monitoring point; v h,i-1 represents the precipitation at the time of the i-1th water level data of the hth upstream monitoring point; w i represents the i-th water level data of the target monitoring point; w' i Represents the water level data of the i-th water level data of the target monitoring point at the same time last year; || is the absolute value function; exp[] is an exponential function with a natural constant as the base. This embodiment uses exp[-] to present the inverse proportional relationship and normalization processing. The implementer can set the inverse proportional function and normalization function according to the actual situation.
[0077] What needs to be explained is that the desirability of the i-th water level data P" i The larger the value, the higher the single importance of the i-th water level data; the difference between the i-th water level data and the water level data at the same time last year |w i -w' i The larger the value, the more attention should be paid to the impact of rainfall. If the rainfall at the upstream monitoring point closer to the target monitoring point is too large, the impact on the water level data will be greater. The larger the value, the The greater the weight.
[0078] According to the single importance of all water level data of the target monitoring point, the corresponding calculation formula for the comparative importance of each water level data of the target monitoring point is obtained as follows:
[0079] K” i =norm(K' i+1 +K' i-1 )×K' i
[0080] In the formula, K” i Indicates the comparative importance of the i-th water level data of the target monitoring point; K' i+1 Indicates the single importance of the i+1th water level data of the target monitoring point; K' i-1 Indicates the single importance of the i-1th water level data of the target monitoring point; K' i It represents the single importance of the i-th water level data of the target monitoring point; norm() is a linear normalization function that normalizes the data value to the interval [0,1].
[0081] It should be noted that since the water level data does not change independently, the higher the single importance of the two adjacent water level data of a certain water level data of the target monitoring point, the higher their comparative importance; the first and last water level data are not analyzed.
[0082] Step S004: Take each water level data of the target monitoring point as each data of the initial dictionary; obtain the coding position front rate of each data according to the comparative importance of all data in the initial dictionary and all water level data of the target monitoring point; and transmit data according to the coding position front rate of all data.
[0083] All single data are extracted, and the coding position rate of single characters is determined according to the number of times they appear and the comparison of importance, and the initial dictionary is constructed; next, the data is traversed, and the coding is flipped by combining the emphasis of the data with the comparison of the previous data in the dictionary table, so that the characters with higher emphasis have smaller codes, reducing the transmission risk.
[0084] Each water level data of the target monitoring point is used as each data of the initial dictionary. In the initial dictionary, the same data is recorded as a data class;
[0085] According to the number of data in all data classes of the initial dictionary and the comparative importance of all water level data of the target monitoring point, the corresponding calculation formula for the coding position front rate of each data is obtained as follows:
[0086]
[0087] In the formula, Zs Indicates the encoding position rate of the sth data in the initial dictionary; m s Indicates the number of data in the data class where the sth data in the initial dictionary is located; K" s,y It indicates the comparative importance of the water level data of the target monitoring point corresponding to the yth data in the data class where the sth data in the initial dictionary is located.
[0088] It should be noted that, for the sth data, the more data there are in the data class where the sth data is located in the initial dictionary, the closer it should be to the front and have a smaller code, so the coding position rate of the sth data is higher.
[0089] Thus, the encoding position front rate of all data is obtained.
[0090] The LZW algorithm is a general data compression algorithm that achieves compression by building a dictionary, replacing repeated patterns in the input sequence with shorter marker codes to achieve data compression. Combined with the encoding position rate of all data, the LZW algorithm is used to perform the following operations on all data:
[0091] For all data, the initial dictionary table can be obtained by forwarding the data with high leading rate and backwarding the data with low leading rate. Then all water level data are traversed to expand the dictionary table. Here, a new encoding is required for a combination data that has not appeared and added to the dictionary table. At this time, by comparing the average leading rate, it is determined whether the Mth combination data of the dictionary needs to flip its encoding with the M-1 combinations of the dictionary.
[0092] According to the coding position rate of all data from high to low, all data are encoded from small to large, and an initial dictionary table is obtained, and all data are recorded as encoded data;
[0093] The acquisition of water level data is a continuous process. For several types of data that have not been encoded into the initial dictionary table, the encoding position front rate of all unencoded data is obtained in the above manner;
[0094] For any uncoded data, when the coding position front rate of the uncoded data is greater than the coding position front rate of the coded data, the uncoded data is inserted before the coded data to obtain the coding of the coded data, and the coded data and all the coded data after the coded data are updated;
[0095] According to the above method, each uncoded data is compared one by one in the direction of decreasing code starting from the coded data with the largest code, until the coding position front rate of the uncoded data is less than or equal to the coding position front rate of the coded data; thereby, all uncoded data are expanded to the initial dictionary table, an extended dictionary table is obtained, and the extended dictionary table is transmitted to the database, so that more important data is encoded first in the extended dictionary table, thereby improving the security of important data transmission.
[0096] So far, the present invention is completed.
[0097] In summary, in an embodiment of the present invention, all water level data, precipitation and water flow of the target monitoring point and the water level data of the upstream monitoring point are obtained, and the stability and rainfall rate of each water level data of the target monitoring point are calculated according to all water level data and precipitation of the target monitoring point, so as to obtain the desirability of each water level data, and the single importance and comparative importance of each water level data of the target monitoring point are calculated in combination with the water level data of the upstream monitoring point, and each water level data of the target monitoring point is used as each data of the initial dictionary, and the comparative importance of all water level data is combined to finally obtain the coding position front rate of each data, and the data is transmitted in combination with the LZW algorithm. The present invention dynamically updates the dictionary table of the LZW algorithm based on the importance of the data, so that the important data is closer to the front of the coding table, ensuring the safety during transportation.
[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A data migration optimization transmission method, characterized in that: The method comprises the following steps: Obtain the water level data of the target monitoring point at each moment, the precipitation and water flow at each water level data moment, and the water level data at the same moment last year at each water level data moment; obtain the water level data of all upstream monitoring points at each moment and the precipitation at each water level data moment; obtain the distance from the target monitoring point to each upstream monitoring point; According to all the water level data of the target monitoring point, the stability of each water level data of the target monitoring point is obtained; according to the precipitation at the time when all the water level data of the target monitoring point are located, the rainfall rate at the time when each water level data of the target monitoring point is located is obtained; according to the stability of each water level data of the target monitoring point, the rainfall rate at the time, the precipitation at the time when all the water level data are located, and the water flow, the desirability of each water level data of the target monitoring point is obtained; According to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year when each water level data of the target monitoring point is located, the single importance of each water level data of the target monitoring point is obtained; according to the single importance of all water level data of the target monitoring point, the comparative importance of each water level data of the target monitoring point is obtained; Each water level data of the target monitoring point is used as each data of the initial dictionary; according to the comparative importance of all the data of the initial dictionary and all the water level data of the target monitoring point, the coding position front rate of each data is obtained; according to the coding position front rate of all the data, data transmission is performed.
2. The data migration optimization transmission method according to claim 1, characterized in that: The method of obtaining the stability of each water level data of the target monitoring point according to all water level data of the target monitoring point includes the following specific steps: The a most recent water level data of each water level data of the target monitoring point in time are used as the comparison water level data of each water level data, where a is the preset number of comparison data; According to each water level data of the target monitoring point and all the comparison water level data of each water level data, the stationarity of each water level data of the target monitoring point is obtained.
3. The data migration optimization transmission method according to claim 2, characterized in that: The specific calculation formula for obtaining the stability of each water level data of the target monitoring point based on each water level data of the target monitoring point and all the comparison water level data of each water level data is: Where P' i Indicates the stability of the i-th water level data of the target monitoring point; a i is the number of comparison water level data of the i-th water level data of the target monitoring point; w i,j represents the jth comparison water level data of the i-th water level data of the target monitoring point; μ i Represents the average value of all comparison water level data of the i-th water level data of the target monitoring point; exp() is an exponential function with a natural constant as the base.
4. The data migration optimization transmission method according to claim 2, characterized in that: The method of obtaining the rainfall rate at the moment of each water level data of the target monitoring point according to the precipitation at the moment of all water level data of the target monitoring point includes the following specific steps: The normalized value of the average precipitation at the time when all the comparison water level data of each water level data of the target monitoring point are located is recorded as the rainfall rate at the time when each water level data of the target monitoring point is located.
5. The data migration optimization transmission method according to claim 1, characterized in that: The method of obtaining the desirability of each water level data at the target monitoring point according to the stability of each water level data at the target monitoring point, the rainfall rate at the time, the precipitation and water flow at the time of all water level data includes the following specific steps: The water level data of the target monitoring point is used as the horizontal axis and the precipitation as the vertical axis to construct a two-dimensional coordinate system of precipitation; In the two-dimensional precipitation coordinate system, the least square method is used to perform a straight line fitting on the data points corresponding to each water level data of the target monitoring point and all its comparison water level data, and the fitting precipitation corresponding to each water level data of the target monitoring point is obtained; The water level data of the target monitoring point is used as the horizontal axis and the water flow rate is used as the vertical axis to construct a two-dimensional coordinate system of water flow rate; In the two-dimensional coordinate system of water flow, the least square method is used to perform straight line fitting on the data points corresponding to each water level data of the target monitoring point and all the comparative water level data, so as to obtain the fitted water flow corresponding to each water level data of the target monitoring point; According to the stationarity of each water level data of the target monitoring point, the rainfall rate at the time, the corresponding fitted precipitation and fitted water flow, the desirability of each water level data of the target monitoring point is obtained.
6. A data migration optimization transmission method according to claim 5, characterized in that: The specific calculation formula for obtaining the desirability of each water level data at the target monitoring point according to the stability of each water level data at the target monitoring point, the rainfall rate at the time, the corresponding fitted precipitation and the fitted water flow is: In the formula, P i Indicates the desirability of the i-th water level data of the target monitoring point; L i represents the rainfall rate at the moment when the i-th water level data of the target monitoring point is located; b represents the preset rainfall rate threshold; P' i represents the stability of the i-th water level data of the target monitoring point; v i Indicates the precipitation at the time of the i-th water level data of the target monitoring point; v' i represents the fitted precipitation corresponding to the i-th water level data of the target monitoring point; r i Indicates the water flow at the time when the i-th water level data of the target monitoring point is located; r' i The fitted water flow corresponding to the i-th water level data of the target monitoring point; | | is the absolute value function; exp[ ] is the exponential function with the natural constant as the base.
7. The data migration optimization transmission method according to claim 1, characterized in that: The specific calculation formula for obtaining the single importance of each water level data of the target monitoring point according to the desirability of each water level data of the target monitoring point, the precipitation at the time when all water level data of all upstream monitoring points are located, the distance from the target monitoring point to each upstream monitoring point, and the water level data at the same time last year at the time when each water level data of the target monitoring point is located is: In the formula, K' i Indicates the single importance of the i-th water level data of the target monitoring point; P" i represents the desirability of the i-th water level data of the target monitoring point; m represents the number of upstream monitoring points; D h represents the distance from the hth upstream monitoring point to the target monitoring point; v h,i+1 represents the precipitation at the time of the i+1th water level data of the hth upstream monitoring point; v h,i represents the precipitation at the time of the i-th water level data of the h-th upstream monitoring point; v h,i-1 represents the precipitation at the time of the i-1th water level data of the hth upstream monitoring point; w i represents the i-th water level data of the target monitoring point; w' i It represents the water level data of the i-th water level data of the target monitoring point at the same time last year; | is the absolute value function; exp[ ] is the exponential function with the natural constant as the base.
8. The data migration optimization transmission method according to claim 1, characterized in that: The specific calculation formula for obtaining the comparative importance of each water level data of the target monitoring point according to the single importance of all water level data of the target monitoring point is: K” i =norm(K’ i+1 +K’ i-1 )×K’ i In the formula, K” i Indicates the comparative importance of the i-th water level data of the target monitoring point; K' i+1 Indicates the single importance of the i+1th water level data of the target monitoring point; K' i-1 Indicates the single importance of the i-1th water level data of the target monitoring point; K' i It represents the single importance of the i-th water level data of the target monitoring point; norm( ) is the linear normalization function.
9. The data migration optimization transmission method according to claim 1, characterized in that: The step of obtaining the coding position front rate of each data according to the comparison importance of all data in the initial dictionary and all water level data of the target monitoring point includes the following specific steps: In the initial dictionary, the same data is recorded as a data class; According to the number of data in all data classes of the initial dictionary and the comparative importance of all water level data of the target monitoring point, the coding position front rate of each data is obtained.
10. A data migration optimization transmission method according to claim 9, characterized in that: The step of obtaining the coding position front rate of each data according to the number of data in all data classes of the initial dictionary and the comparative importance of all water level data of the target monitoring point includes the following specific steps: The product of the number of data in the data class where each data in the initial dictionary is located and the sum of the comparative importance of the water level data of the target monitoring point corresponding to all the data in the data class where each data in the initial dictionary is located is recorded as the encoding position front rate of each data.
Citation Information
Patent Citations
Smart community face data storage method and system
CN116521093A
Smart city monitoring system and data processing method
CN117113236A
Wireless sensor network data intelligent compression method
CN117376429A
Visual surrounding underground water level monitoring and early-warning threshold design method for a deep foundation pit
WO2022111518A1
Data compression method and apparatus, movable platform, and storage medium
WO2022198483A1
