A weather forecast data compression method based on video compression technology

Abstract

The present invention discloses a weather forecast data compression method based on video compression technology, belonging to the field of data compression technology. The steps include: S1, obtaining original weather forecast data, extracting time slices and weather element data; S2, preprocessing weather forecast data: mapping the same weather element data of each station at the same time to the interval [0-1023*n], n is the number of mapped channels; S3, three-dimensional grid point reconstruction of weather data: storing weather forecast data according to three-dimensional grid point data, and forming a weather data map in the data format of three color channels of the corresponding picture; S4, using a video compression tool to perform lossless compression encoding of weather forecast data with a bit depth of 10 bits and a chrominance-luminance ratio of 4:4:4; S5, using a video compression tool to decode the weather data of the encoded file generated by S4, and perform inverse transformation of weather forecast data. The present invention reduces the storage space of weather forecast data, and the final compression ratio can reach 10:1.

Description

Technical Field

[0001] The invention belongs to the technical field of data compression, and in particular relates to a weather forecast data compression method based on video compression technology. Background Art

[0002] Weather forecast data has the characteristics of large data volume, many types of meteorological elements, and high timeliness. Since a large amount of weather forecast data is spatiotemporal data, which records the observed quantities of various physical quantities within the time and space range, these massive amounts of weather data consume a lot of storage resources and network transmission resources. Therefore, choosing a good weather forecast data compression algorithm can greatly reduce disk space usage, and at the same time the network load will be smaller, improving user experience satisfaction. Weather forecast data includes 6 meteorological elements:

[0003] Storage Channel Meteorological elements unit PRE Total precipitation in the past hour mm TMP 2m temperature K PRS Surface pressure Pa RH Relative humidity ％ U10 10 metre U wind component m / s V10 10metre v wind component m / s

[0004] Video compression has a long history of development and has formed a very mature and high-compression algorithm. The composition of meteorological data is similar to that of video data. Meteorological data generated by numerical forecasting models are generally organized in grid data, with one grid point predicting one value. Therefore, each grid point in the meteorological data corresponds to each pixel in the video. The meteorological data can be reconstructed into a frame in the video sequence corresponding to the three-dimensional grid data, and the video compression tool can be used to compress the meteorological forecast data.

[0005] In [Zhao Suxuan, Luo Jian, Yang Chengyin. Lossless compression method of meteorological grid data based on BP neural network [J] Progress in Earth Science, 2008, 23 (2): 637-642.], compression technology can be roughly divided into three types: lossless, lossy and near-lossless. Lossless compression can retain all the information in the data, but the compression ratio is generally not high, usually 2:1 to 5:1; for a large amount of meteorological forecast data, a sufficiently large compression ratio is necessary, but data loss will have an immeasurable impact on the accuracy and quality of data prediction. Summary of the invention

[0006] Purpose of the invention: In view of the above problems, the present invention proposes a weather forecast data compression method based on video compression technology, which processes the weather forecast data into data in video format, thereby realizing the compression of the weather forecast data and improving its compression ratio.

[0007] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: a weather forecast data compression method based on video compression technology, comprising the following steps:

[0008] S1. Obtain original weather forecast data and extract time slices and weather element data;

[0009] S2. Preprocess the weather forecast data: map the same meteorological element data of each station at the same time to the interval [0~1023*n], where n is the number of mapping channels;

[0010] S3, meteorological data three-dimensional grid reconstruction: the meteorological forecast data is stored as three-dimensional grid data, and the meteorological data graph is formed in the data format of the three color channels of the corresponding picture;

[0011] S4, using a video compression tool to perform lossless compression encoding of the weather forecast data with a bit depth of 10 bits and a chrominance-luminance ratio of 4:4:4;

[0012] S5. Use a video compression tool to decode the meteorological data of the encoded file generated by S4, and perform an inverse transformation of the meteorological forecast data.

[0013] Preferably, the mapping method in S2 is as follows:

[0014] Assume that the data values ​​of a meteorological element before and after mapping are x and y respectively, the accuracy of the meteorological element to be retained is a, the maximum value of the meteorological element is max, and the minimum value is min, then the mapping expression is:

[0015]

[0016] After all meteorological element data are mapped, a meteorological forecast data frame sequence is obtained.

[0017] Preferably, S3 stores the weather forecast data as three-dimensional grid data in the following manner:

[0018] Using the .mat file of the matlab tool and referring to the three color channel data formats of the image, three different meteorological elements are combined into a three-dimensional meteorological forecast data image;

[0019] Suppose meteorological data 1 is recorded as X1, meteorological data 2 is recorded as X2, and meteorological data 3 is recorded as X3, then the MATLAB command for the three-dimensional meteorological forecast data is: cat(3,X1,X2,X3).

[0020] Preferably, the S4 uses the HM tool of the HEVC standard to perform 10-bit lossless compression encoding on the weather forecast data graph; the command parameters in the encoding configuration file are set as follows:

[0021] InputChromaFormat:444;

[0022] TransquantBypassEnable = 1;

[0023] CUTransquantBypassFlagForce=1.

[0024] Preferably, the S5 data inverse transformation method is as follows:

[0025] Suppose the data after decoding and inverse transformation of a certain meteorological element is k′, the value after mapping is y, the maximum value of the meteorological element is max, a is the accuracy that needs to be retained for the meteorological element, n is the number of mapped channels, and the maximum allowable error is error. Then the inverse transformation formula is:

[0026]

[0027] sqrt(mse(yk′))≤error

[0028] Where sqrt is the square root function and mse is the mean square error function.

[0029] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0030] The weather forecast data compression method based on video compression proposed by the method of the present invention maps the weather forecast data of each station in the same time period to the range of 10-bit image storage, forms a weather forecast data sequence from the weather forecast data frame according to the time dimension, and then uses video compression technology to encode and compress the weather forecast data frame sequence, thereby reducing the storage space of the weather forecast data, greatly facilitating the transmission of the weather forecast data, and at the same time retaining the weather forecast data information as much as possible, with a compression ratio of up to 10:1. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of weather forecast data compression of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0033] like Figure 1 As shown, the present invention proposes a weather forecast data compression method based on video compression, which specifically includes:

[0034] (1) Reading original weather forecast data: Read the original ground weather data, extract the time slice and weather element data, save the same weather element data of each station at the same time in the form of a two-dimensional matrix, and obtain the weather forecast data frame.

[0035] (2) Weather forecast data frame preprocessing. The same weather data of each station at the same time is mapped to the interval [0~1023*n] (10 bit depth corresponds to 0~1023 data range) according to formula (1), where n is the number of mapped channels. In order to reduce the error caused by data mapping, when the data range is too large, it can be divided into n channels. Specific mapping method:

[0036] Assume a meteorological element data k, the accuracy of the meteorological element to be retained is a (if the retained accuracy is 0.1, then multiply the value by 10 and round it, that is, a = 10), the maximum value of the meteorological element is max (retained accuracy), the minimum value is min (retained accuracy), the data values ​​before and after mapping are x, y respectively, then the mapping method is:

[0037]

[0038] After all meteorological element data are mapped, a meteorological forecast data frame sequence is obtained.

[0039] (3) Reconstruction of three-dimensional grid points of meteorological data: The meteorological forecast data is stored as three-dimensional grid point data, and the three color channel data formats of the corresponding pictures are used to form a meteorological data image; that is, the three different meteorological elements are combined into a three-dimensional meteorological forecast data image. Specifically:

[0040] Using the .mat file of the matlab tool and referring to the three color channel data formats of the image, three different meteorological elements are combined into a three-dimensional meteorological forecast data image;

[0041] Suppose meteorological data 1 is recorded as X1, meteorological data 2 is recorded as X2, and meteorological data 3 is recorded as X3. For example, the total precipitation in the past hour is recorded as X1, the surface pressure is recorded as X2, and the relative humidity is recorded as X3. The matlab command that constitutes the three-dimensional meteorological forecast data is: cat(3,X1,X2,X3).

[0042] (4) Use the HM tool of the HEVC (High Efficiency Video Coding) standard (a new video compression standard) to perform lossless compression encoding of the weather forecast data graph with a bit depth of 10 bits and a chroma-luminance ratio of 4:4:4; enable Transform Quantizer Bypass (TQB) in the encoding configuration file. Set the command parameters in the encoding configuration file:

[0043] –InputChromaFormat:444;

[0044] –TransquantBypassEnable=1;

[0045] –CUTransquantBypassFlagForce=1.

[0046] The encoding process in this embodiment is implemented as follows:

[0047] TAppEncoder[--help][-c config.cfg][--parameter=value]

[0048] TAppEncoder is the encoder, help is the help parameter, -c config.cfg is the specified encoding configuration file, --parameter=value is the specific parameter value set in the encoding configuration file, and the process will output a binary stream file with bin as the suffix.

[0049] (5) Use a video compression tool to decode the encoded file (compressed data stream (bin file)) generated in step (4) and perform data inverse transformation.

[0050] The decoding process in this embodiment is implemented as follows:

[0051] TAppDecoder -b str.bin -o dec.yuv

[0052] Where TAppDecoder is the decoder, -b str.bin is the output result of step (4), and -o dec.yuv is the decoded meteorological data map.

[0053] Suppose the data after decoding and inverse transformation of a certain meteorological element is k′, the value after mapping is y, the maximum value of the meteorological element is max, a is the accuracy that needs to be retained for the meteorological element, n is the number of mapped channels, and the maximum allowable error is error. Then the inverse transformation formula is:

[0054]

[0055] sqrt(mse(yk′))≤error

[0056] Where sqrt is the square root function and mse is the mean square error function.

[0057] The following table is a calculation diagram of the weather forecast data compression error of the present invention. Storage channel represents the variable name corresponding to different meteorological elements in the .mat two-dimensional matrix storage file, meteorological element represents a specific meteorological element, error represents the error between the compressed data and the uncompressed data by the method of the present invention, and the average compression ratio is the size of the compressed data / the original data.

[0058]

[0059] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A weather forecast data compression method based on video compression technology, characterized in that: The method includes: S1. Obtain original weather forecast data and extract time slices and weather element data; S2. Preprocess the weather forecast data: map the same weather element data of each station at the same time to the interval [0~1023*n], where n is the number of mapped channels; the mapping method is as follows: Assume that the data values ​​of a meteorological element before and after mapping are x and y respectively, the accuracy of the meteorological element to be retained is a, the maximum value of the meteorological element is max, and the minimum value is min, then the mapping expression is: After all meteorological element data are mapped, a meteorological forecast data frame sequence is obtained; S3. Reconstruction of three-dimensional grid points of meteorological data: The meteorological forecast data is stored as three-dimensional grid point data, and a meteorological data graph is formed in the data format of the three color channels of the corresponding image. The method is as follows: Using the .mat file of the matlab tool and referring to the three color channel data formats of the image, three different meteorological elements are combined into a three-dimensional meteorological forecast data image; Assume that meteorological data 1 is recorded as X1, meteorological data 2 is recorded as X2, and meteorological data 3 is recorded as X3. The MATLAB command for the three-dimensional meteorological forecast data is: cat(3,X1,X2,X3); S4, using a video compression tool to perform lossless compression encoding of the weather forecast data with a bit depth of 10 bits and a chrominance-luminance ratio of 4:4:4; S5. Decode the meteorological data of the encoded file generated by S4 using a video compression tool, and perform an inverse transformation of the meteorological forecast data, as follows: Assume that the data after decoding and inverse transformation of a meteorological element is k′, the value after mapping is y, the maximum value of the meteorological element is max, a is the accuracy that needs to be retained for the meteorological element, n is the number of mapped channels, and the maximum allowable error is error. Then the inverse transformation formula is: sqrt(mse(yk′))≤error Where sqrt is the square root function and mse is the mean square error function.

2. The weather forecast data compression method based on video compression technology according to claim 1, characterized in that: The S4 uses the HM tool of the HEVC standard to perform 10-bit lossless compression encoding on the weather forecast data graph; the command parameters in the encoding configuration file are set as follows: InputChromaFormat:444; TransquantBypassEnable = 1; CUTransquantBypassFlagForce=1.

Images