Radar-based data compression method, decoding method, device, storage medium and product

Abstract

The present invention discloses a radar-based data compression method, decoding method, device, storage medium and product. The compression method comprises dividing the original radar-based data into blocks according to the number of threads to obtain a plurality of original data blocks; determining the size of the compression cache area according to the size of the original radar-based data, and setting the compressed data block corresponding to the original data block at the offset position of the compression cache area according to the size of each original data block; using multiple threads to simultaneously compress all original data blocks, and writing the compressed data blocks into the corresponding positions of the compression cache area respectively; determining the compression storage space according to the number and size of the compressed data blocks and the size of the data description header, writing the data description header and all the compressed data blocks into the compression storage space in sequence, and then merging to obtain the radar-based data compression result. The present invention greatly improves the compression efficiency of radar-based data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a parallel dual-polarization weather radar-based data compression method, decoding method, device, storage medium and product. Background Art

[0002] Weather radar base data is the observation data recorded by the weather radar in a complete volume scan. Each base data of a traditional Doppler weather radar is about 10MByte. With the introduction of dual polarization technology and the improvement of resolution, the base data of a dual polarization Doppler weather radar can reach 120MByte or even higher. It takes about 5 to 6 minutes for an ordinary mechanical scanning Doppler weather radar to complete a volume scan, while a one-dimensional phased array Doppler weather radar only needs 1 minute to complete the scan. A weather radar base data can generate a variety of meteorological products after a series of processing.

[0003] When processing multiple radar base data, the base data needs to be compressed, transmitted, and decompressed. Currently, weather radars generally use a single Bzip2 algorithm (data compression algorithm). Based on the AMD Ryzen 9 7900X CPU, it takes 1 to 2 seconds to decompress a compressed 28MB base data to 120MB, and compression takes longer. At the same time, the observation value of the radar base data is also lossy encoded, and one data is saved per Byte. In actual use, it is usually necessary to decode it into a floating point number for calculation. The format of radar base data is mostly stored in radial stream format. Usually, streaming decoding belongs to serial processing. Decoding a CINRAD-SA model (SA represents S-band A model radar) base data is usually between 1 and 2 seconds. The conversion of phased array radar base data takes longer time. The compression efficiency and decoding efficiency of the front end have a great impact on the processing time of radar products. Summary of the invention

[0004] The object of the present invention is to provide a radar-based data compression method, decoding method, device, storage medium and product to solve at least one of the problems of low efficiency of traditional weather radar front-end data compression, decompression and decoding processes.

[0005] The present invention solves the above technical problems through the following technical solutions: a radar-based data compression method, comprising:

[0006] Obtaining binary-coded original radar base data; wherein the original radar base data includes a base data file header corresponding to the radar base data and all radial data encoding observation values ​​as single-byte integer values, wherein the base data file header includes the number of plane scans, the number of radial data contained in each plane scan, observation elements, the number of radial sampling points of each observation element, and the site information of the radar, and electromagnetic parameters during observation;

[0007] Dividing the storage area of ​​the original radar base data into blocks according to the number of CPU threads to obtain a plurality of original data blocks;

[0008] Applying for a compression cache area according to the size of the original radar base data, and setting a compressed data block corresponding to the original data block at an offset position in the compression cache area according to the size of each original data block;

[0009] Using multiple threads to compress all original data blocks at the same time, and writing the compressed data streams into corresponding positions of the compression cache area respectively; wherein each thread corresponds to the compression processing of one original data block;

[0010] The compression storage space is determined according to the size of each compressed data block obtained by compression and the size of the data description header, the data description header and all the compressed data blocks are sequentially written into the compression storage space, and then the combined radar-based data compression result is obtained.

[0011] Furthermore, each thread uses bzip2, gzip or 7zip algorithm to compress the original data block to obtain a corresponding compressed data block.

[0012] Furthermore, the data description header includes a compressed data identifier, the number of compressed data blocks, a compression method, the size of each compressed data block, and the size of each original data block.

[0013] Furthermore, the calculation formula for the size of the data description header is:

[0014] S d =A1+A2+A3+A4+A5×n;

[0015] Among them, S d Indicates the size of the data description header, A1 indicates the number of bytes occupied by the compressed data identifier, A2 indicates the number of bytes occupied by the version number, A3 indicates the number of bytes occupied by the number of compressed data blocks, A4 indicates the number of bytes occupied by the compression method, A5 indicates the number of bytes occupied by the size of an original data block and its corresponding compressed data block size, and n indicates the number of compressed data blocks.

[0016] Based on the same concept, the present invention also provides a radar-based data decoding method, comprising:

[0017] Receiving a radar-based data compression result obtained by using the radar-based data compression method as described above; wherein the radar-based data compression result includes a data description header and each compressed data block;

[0018] Decompressing the radar-based data compression result to obtain decompressed data consistent with the binary-coded original radar-based data;

[0019] Constructing a container object, and transferring the decompressed data into the container object;

[0020] Using the container object to read the file header from the decompressed data, basic information is obtained; wherein the basic information includes the number of plane scans, the number of radial data contained in each plane scan, observation elements, the number of radial sampling points of each observation element, the site information of the radar, and the electromagnetic parameters during observation;

[0021] Constructing a plane scan list and a radial data list according to the basic information;

[0022] Constructing a decoding data storage space of the plane scan list and the radial data list in the container object, and recording the basic information and each plane scan information in a corresponding position of the dictionary space of the base data information and the plane information in the container object;

[0023] Scan the radial data area of ​​the decompressed storage space, determine the starting address of each radial data according to the size of each radial data, and then obtain the starting address of each radial data in the decompressed storage space, and construct a starting address list as an index;

[0024] Based on the starting address list and index, each radial data is processed in parallel using multiple threads, each sampling point of each observed element on the radial direction is converted into a floating-point value, and the converted floating-point value is stored in the storage location of the corresponding plane scan, corresponding radial direction, corresponding element, and corresponding sampling point of the decoding data storage space in the container object.

[0025] Further, the radar-based data compression result is decompressed, specifically including:

[0026] Parsing the radar-based data compression result to obtain a data description header;

[0027] Determine the size of the original radar base data according to the number of compressed data blocks in the data description header and the size of the original data block;

[0028] Determine the decompression storage space according to the size of the original radar base data;

[0029] Constructing a source address list in the compressed data area according to the size of each compressed data block in the data description header; constructing a target address list in the decompressed storage space according to the size of each original data block in the data description header;

[0030] Based on the compression method in the data description header, multiple threads are used to simultaneously decompress all compressed data blocks, and the compressed data blocks are decompressed from the source address list to the target address list to obtain decompressed data blocks;

[0031] According to the decompressed data blocks, decompressed data consistent with the binary-coded original radar base data is generated in the decompressed storage space.

[0032] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the radar-based data compression method or the radar-based data decoding method as described above.

[0033] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the radar-based data compression method or the radar-based data decoding method as described above is implemented.

[0034] Based on the same concept, the present invention also provides a computer program product, including a computer program / instruction, which implements the radar-based data compression method or radar-based data decoding method as described above when executed by a processor.

[0035] Beneficial Effects

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] The present invention uses a parallel strategy to accelerate the compression and decompression of radar-based data through the multi-core performance of the CPU, thereby greatly improving the compression and decompression efficiency of the radar-based data; the present invention also parses the decompressed data from a binary data stream into a container object mode of a floating-point value suitable for calculation, thereby reducing the conversion process in the subsequent radar data processing process, and greatly accelerating the front-end processing of the radar data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1is a flow chart of a radar-based data compression method according to an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of data size during compression in an embodiment of the present invention;

[0041] Figure 3 is a flow chart of a radar-based data decoding method according to an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of data size during the decompression process in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0045] Example 1

[0046] like Figure 1 and Figure 2 As shown, a radar-based data compression method provided by an embodiment of the present invention includes the following steps:

[0047] Step A1: Get binary-coded raw radar-based data.

[0048] A continuous binary-coded raw radar base data is generated through a volume scan observation of a dual-polarization weather radar. The raw radar base data includes a base data file header that describes the basic information of the base data and all radial data that encode the observation value as a single-byte integer value; the base data file header includes the number of plane scans corresponding to the radar base data, the number of radial data contained in each plane scan, the observation elements, the number of radial sampling points for each observation element, as well as the radar site information and the electromagnetic parameters during observation.

[0049] Step A2: The compressor divides the storage area of ​​the original radar-based data into blocks according to the number of CPU threads to obtain the starting address of each data block, that is, each logical original data block.

[0050] The number of original data blocks must be smaller than the number of CPU threads. For example, if the number of CPU threads is 24, then the number N of original data blocks is smaller than 24, for example, the number N of original data blocks is 20.

[0051] Step A3: Determine a compression cache area according to the size of the original radar base data, and set a compressed data block corresponding to each original data block at an offset position in the compression cache area according to the size of the original data block.

[0052] According to the number and size of the original data blocks obtained in step A2, a cache area consistent with the size of the original radar base data is applied as a compressed cache area, and an offset position of the output is located for each compressed data block according to the size of the original data block, so that when the original data block is compressed, the obtained compressed data stream can be directly written to the corresponding position of the compressed cache area without overflow errors.

[0053] For example, Figure 2 As shown in the figure, assuming that the size of the original radar base data is 2566 bytes, the original radar base data is divided into 4 original data blocks, then the sizes of the 4 original data blocks (i.e., uncompressed blocks) are 640 bytes, 640 bytes, 640 bytes, and 646 bytes respectively. The number of blocks in the compression cache area is the same as the number of original data blocks. The offset position of each compressed data block is the sum of the offset positions of the previous original data blocks, that is, the number of blocks in the compression cache area is 4, each block corresponds to the size of 4 original data blocks, and the 4 offset positions are offset by 0, 640, 1280, and 1920 on the initial address of the compression cache area, that is, the size of the space that can be used for the output is 640 bytes, 640 bytes, 640 bytes, and 646 bytes respectively, to ensure that there is no overflow when compression fails.

[0054] Step A4: The compressor uses multiple threads to simultaneously compress all original data blocks, and writes the compressed data blocks obtained by compression into corresponding positions in the compression cache area respectively.

[0055] Exemplarily, assuming that the number of original data blocks is 4, the compressor uses 4 threads to compress the 4 original data blocks at the same time, and each thread compresses one original data block. The use of threads to compress data blocks in parallel greatly improves the compression efficiency of radar-based data. The 4 compressed data blocks obtained by data compression are respectively stored in the corresponding positions of the compression cache area. Exemplarily, the sizes of the 4 original data blocks are 640 bytes, 640 bytes, 640 bytes, and 646 bytes, and the sizes of the 4 compressed data blocks obtained by compression are 64 bytes, 100 bytes, 124 bytes, and 146 bytes, respectively. These 4 compressed data blocks are respectively located in the corresponding positions of the compression cache area, and the space of each block in the compression cache area that is larger than the size of the compressed data block is free space.

[0056] In a specific implementation of the present invention, each thread uses bzip2, gzip or 7zip algorithm to compress the original data block to obtain a corresponding compressed data block.

[0057] Step A5: Determine the compression storage space according to the number and size of the compressed data blocks and the size of the data description header, write the data description header and all the compressed data blocks into the compression storage space in sequence, and then obtain the combined radar-based data compression result.

[0058] The size of the compressed storage space can be determined by applying for a continuous storage space from the memory again according to the number and size of the compressed data blocks and the size of the data description header. The size of the compressed storage space is equal to the sum of the size of each compressed data block and the size of the data description header. The data description header includes information such as the compressed data identifier, the number of compressed data blocks, the compression method, the size of each compressed data block, and the size of each original data block.

[0059] Exemplarily, the sizes of the four compressed data blocks obtained by compression are 64 bytes, 100 bytes, 124 bytes, and 146 bytes respectively, and the size of the compressed storage space is equal to the sum of 434 bytes and the size of the data description header.

[0060] In a specific implementation of the present invention, the calculation formula for the size of the data description header is:

[0061] S d =A1+A2+A3+A4+A5×n;

[0062] Among them, S dIndicates the size of the data description header, A1 indicates the number of bytes occupied by the compressed data identifier, A2 indicates the number of bytes occupied by the version number, A3 indicates the number of bytes occupied by the number of compressed data blocks, A4 indicates the number of bytes occupied by the compression method, A5 indicates the number of bytes occupied by the size of an original data block and its corresponding compressed data block size, and n indicates the number of compressed data blocks.

[0063] In this embodiment, the compressed data identifier is a string of 4 bytes, fixed as "PZCO", the version number occupies 4 bytes, the number of bytes occupied by the compressed data block is 4, the number of bytes occupied by the compression method is 8, and the number of bytes occupied by the size of an original data block and its corresponding compressed data block is 16 (i.e. 8+8). d =20+16×n.

[0064] First, write the data description header into the compressed storage space, then copy each compressed data block into the compressed storage space in sequence, and release the compressed cache area.

[0065] Example 2

[0066] like Figure 3 As shown, a radar-based data decoding method provided by the present invention includes:

[0067] Step B1: receiving the radar-based data compression result obtained by using the radar-based data compression method described in Example 1.

[0068] The original radar-based data is compressed to obtain a radar-based data compression result, and the radar-based data compression result is transmitted to the processing module through the network. In a specific embodiment of the present invention, the radar-based data compression result is obtained by using the radar-based data compression method of Example 1 of the present invention, and then the radar-based data compression result in the compressed storage space is transmitted to the processing module in the form of a network data stream or saved as a file and shared. The radar-based data compression result includes a data description header and each compressed data block.

[0069] Step B2: Decompress the radar-based data compression result to obtain decompressed data that is consistent with the binary-coded original radar-based data.

[0070] The processing module loads the radar-based data compression result into the memory, and uses the decompressor to read the data description header and the size of each original data block. Figure 4 As shown, the radar base data compression result is decompressed, specifically including:

[0071] Step B2.1: parse the radar-based data compression result to obtain the data description header.

[0072] Step B2.2: Determine the size of the original radar base data according to the number of compressed data blocks in the data description header and the size of the original data block.

[0073] Step B2.3: Determine the decompressed storage space according to the size of the original radar base data.

[0074] like Figure 4 As shown, the sum of the sizes of the four compressed data blocks is 434 bytes, the sizes of the four compressed data blocks are 64 bytes, 100 bytes, 124 bytes, and 146 bytes respectively, and the sizes of the four original data blocks are 640 bytes, 640 bytes, 640 bytes, and 646 bytes respectively. Therefore, the size of the original radar base data is 2566 bytes, that is, the size of the decompressed storage space is 2566 bytes.

[0075] Step B2.4: construct a source address list in the compressed data area according to the size of each compressed data block in the data description header; construct a target address list in the decompressed storage space according to the size of each original data block in the data description header.

[0076] The source address list includes the starting address of each compressed data block, and the target address list includes the write address of the data stream (i.e., the original data block) obtained by decompressing each compressed data block. The starting address of each compressed data block can be determined according to the size of each compressed data block, and the write address of the decompressed data stream can be determined according to the size of the original data block corresponding to each compressed data block.

[0077] Step B2.5: Based on the compression method in the data description header, use multiple threads to simultaneously decompress all compressed data blocks, decompress the compressed data blocks from the source address list to the target address list, and obtain decompressed data blocks.

[0078] Based on the compression method in the data description header and the starting address of each compressed data block in the source address list, the decompressor points the data input of each decompression thread to the starting address of the corresponding compressed data block, uses multiple threads to decompress all compressed data blocks at the same time, and outputs the decompressed data stream to the corresponding target address.

[0079] Step B2.6: Based on the decompressed data block, generate decompressed data consistent with the binary-encoded original radar base data in the decompressed storage space.

[0080] Step B3: construct a container object and transfer the decompressed data of the decompressed storage space into the container object.

[0081] Step B4: Use the container object to read the file header from the decompressed data to obtain basic information.

[0082] The basic information includes the radar site information, electromagnetic parameter information during observation, the number of plane scans, the number of radial data contained in each plane scan, the observation elements and the number of radial sampling points for each observation element. The observation elements include reflectivity intensity, radial velocity, spectral width, correlation coefficient, differential reflectivity, differential phase shift coefficient and other dual-polarization radar observation elements.

[0083] Step B5: Construct a plane scan list and a radial data list according to the basic information.

[0084] A corresponding plane scan list is constructed according to the plane scan number, the plane scan list is circulated, and a radial data list is constructed for each plane scan according to the radial number in each plane scan list.

[0085] Step B6: Construct the decoding data storage space of the plane scanning list and the radial data list in the container object, and record the basic information and each plane scanning information in the corresponding position of the dictionary space of the base data information and the plane information in the container object.

[0086] The container object stores the plane scanning information in the basic information into the plane scanning attribute, and applies for the decoding data storage space of the corresponding observation quantity for each radial direction of each plane scan according to the number of sampling points of each observation quantity in each plane scanning information. That is, the storage space size of each radial data is equal to the sum of the sizes of all observation elements, and the storage space size of each observation element is equal to the number of radial sampling points of the observation element × the size of the floating point number.

[0087] Step B7: Scan the radial data area of ​​the decompressed storage space, determine the starting address of each radial data according to the size of the radial data, and then obtain the starting address of each radial data in the decompressed storage space, and construct a starting address list as an index.

[0088] The radial data part of the original radar base data in the storage space is scanned and decompressed, and the starting address of each radial data is determined according to the size of each radial data. Then, a starting address list of each radial data contained in the plane scan is constructed for each plane scan and used as an index.

[0089] Step B8: Based on the starting address list and index, use multi-threading to parallel process each radial data, convert each sampling point of each observation element in the radial direction into a floating-point value, and store the converted floating-point value in the corresponding plane scan, corresponding radial, corresponding element, and corresponding sampling point storage location of the decoding data storage space in the container object.

[0090] In the container object, each radar sampling point is located by four parameters: plane scan, radial, element, and sampling point. In the container object, for each plane scan, multi-threaded parallel processing is used to process each radial data to which it belongs. Each radial data is processed by one thread, that is, a certain number of threads N are applied according to the thread capacity of the CPU, and N radials are processed at the same time until all radials are processed, and then the next plane scan is processed until all plane scans are processed. This process converts each sampling point of each observation element on the radial direction from the encoded BYTE integer value to a floating-point value, and stores it in the floating-point data storage of the corresponding plane scan, the corresponding radial, the corresponding element, and the corresponding sampling point position of the container object, and finally obtains a weather radar base data container object with floating-point storage, which is convenient for subsequent data processing.

[0091] The processing of container objects includes data container construction, parameter conversion and data decoding. The data container construction obtains the plane scan number, radial number, observation element and sampling point number of the observation element from the base data file header to construct a complete plane scan list and radial data list, and applies for the corresponding floating point number space for each observation element of each radial data. The parameter conversion converts the volume scan parameters and the parameters of each plane scan into the dictionary of their respective objects, which consists of parameter name and parameter value; finally, the radial data is converted from the encoded data into the decoded floating point observation value.

[0092] Structurally, the container object presents the decompressed data as a set of radial scanning objects in units of multiple radar radial data, and is divided into multiple plane scanning levels with different elevation angles according to the scanning logic. Each observation sampling point of the radial data is a floating-point value.

[0093] The container object is designed as an object-oriented encapsulation container of a volume scan record, and its data hierarchy includes: volume scan, plane scan, and radial unit. The volume scan contains the basic information of the volume scan, such as site parameters, performance parameters, and a plane scan list; each plane scan is a circular scan data of an elevation angle, which records the observation data of the elevation angle and a radial unit list; each radial is a physical storage area for observation data, including azimuth, elevation, and multiple floating-point arrays expressed in the form of a dictionary, which are used to store the sampling data of a radar radial detection. Each observation element applies for a corresponding number of floating-point arrays according to its sampling points to store the data of each sampling point.

[0094] The container object embodies the complete logical structure of radar base data. The container object can construct a decoded data storage space according to the basic information of the radar base data such as the plane scanning number, radial number, and size of the observation elements, and realize data access and operation through the interface. The minimum unit of the data structure in the container object is radial, wherein each radial unit contains several floating-point arrays for storing the data values ​​of each observation element on a scan line, for example, including reflectivity, radial velocity, and spectral width, then the radial unit contains three floating-point arrays of the three observation quantities of reflectivity, radial velocity, and spectral width. A plane scanning object is recorded by a list of several radial units of the same elevation angle, and the plane scanning object also contains various parameters of each plane scanning object recorded in the form of a dictionary. Then, a complete radar base data is composed of a list of several plane scanning objects of different elevation angles, and at the same time contains various parameters of the radar base data recorded in the form of a dictionary. The present invention converts the encoded integer data into decoded floating-point observation values ​​through the container object, reduces the data decoding process in the subsequent processing, accelerates the data processing, and provides an object-oriented base data access interface for the data calculation module.

[0095] In order to illustrate the effectiveness of the present invention, compression and decompression tests with different numbers of data blocks are performed on multiple original radar base data. The size of the original radar base data is about 120 megabytes, which is about 22 to 24 megabytes after compression. The results on the AMD 7900x CPU are shown in Table 1, and the time unit is milliseconds. As can be seen from Table 1, when the number of blocks is more than 8, the decompression time of the data has reached the ideal performance of supporting interactive operations. Compared with the traditional Bzip2 single-block compression method, the present invention can greatly save time.

[0096] Table 1 Compression and decompression results of different numbers of data blocks

[0097]

[0098] The time consumption of decoding and converting radar base data by different methods is tested, and the results are shown in Table 2, where the time unit is milliseconds. As can be seen from Table 2, compared with the traditional non-parallel method, the present invention adopts the optimal radial parallel decompression and conversion in the parallel scheme, which greatly improves the efficiency of decompressing data and reduces the decompression time.

[0099] Table 2 Time results of data decoding conversion by different methods

[0100]

[0101] The present invention adopts parallel processing and reasonable processing strategy design to give full play to the performance of supporting hardware, so that the compression, decompression and decoding of radar-based data can achieve 5 to 8 times the performance of that without parallel optimization processing in a hardware environment with more than 8 cores and 16 threads, greatly saving the processing time of front-end data storage, decompression and decoding.

[0102] Example 3

[0103] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instruction stored in the memory, and the processor executes the computer program / instruction to implement the radar-based data compression method or the radar-based data decoding method in the embodiment of the present application.

[0104] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0105] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0106] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the radar-based data compression method or radar-based data decoding method in the embodiment of the present application.

[0107] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0108] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0109] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the radar-based data compression method or radar-based data decoding method in the embodiment of the present application.

[0110] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A radar-based data decoding method, characterized in that: The decoding method comprises: Obtaining binary-coded original radar base data; wherein the original radar base data includes a base data file header and all radial data encoding observation values ​​as single-byte integer values, wherein the base data file header includes the number of plane scans corresponding to the radar base data, the number of radial data contained in each plane scan, observation elements, the number of radial sampling points of each observation element, and the radar site information, and electromagnetic parameters during observation; Dividing the storage area of ​​the original radar base data into blocks according to the number of CPU threads to obtain a plurality of original data blocks; Applying for a compression cache area according to the size of the original radar base data, and setting a compressed data block corresponding to the original data block at an offset position in the compression cache area according to the size of each original data block; Using multiple threads to compress all original data blocks at the same time, and writing the compressed data streams into corresponding positions of the compression cache area respectively; wherein each thread corresponds to the compression processing of one original data block; Determine the compression storage space according to the size of each compressed data block obtained by compression and the size of the data description header, write the data description header and all the compressed data blocks into the compression storage space in sequence, and then obtain the combined radar-based data compression result; receiving the radar-based data compression result; wherein the radar-based data compression result includes a data description header and each compressed data block; Decompressing the radar-based data compression result to obtain decompressed data consistent with the binary-coded original radar-based data; Constructing a container object, and transferring the decompressed data into the container object; Using the container object to read the file header from the decompressed data, basic information is obtained; wherein the basic information includes the number of plane scans, the number of radial data contained in each plane scan, observation elements, the number of radial sampling points of each observation element, the site information of the radar, and the electromagnetic parameters during observation; Constructing a plane scan list and a radial data list according to the basic information; Constructing a decoding data storage space of the plane scan list and the radial data list in the container object, and recording the basic information and each plane scan information in a corresponding position of the dictionary space of the base data information and the plane information in the container object; Scan the radial data area of ​​the decompressed storage space, determine the starting address of each radial data according to the size of each radial data, and then obtain the starting address of each radial data in the decompressed storage space, and construct a starting address list as an index; Based on the starting address list and index, each radial data is processed in parallel using multiple threads, each sampling point of each observed element on the radial direction is converted into a floating-point value, and the converted floating-point value is stored in the storage location of the corresponding plane scan, corresponding radial direction, corresponding element, and corresponding sampling point of the decoding data storage space in the container object.

2. The radar-based data decoding method according to claim 1, characterized in that: Each thread uses bzip2, gzip or 7zip algorithm to compress the original data block to obtain a corresponding compressed data block.

3. The radar-based data decoding method according to claim 1, characterized in that: The data description header includes a compressed data identifier, the number of compressed data blocks, a compression method, the size of each compressed data block, and the size of each original data block.

4. The radar-based data decoding method according to claim 3, characterized in that: The calculation formula for the size of the data description header is: <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> d <h2 style=";text-align:left;direction:ltr"> (A1, A2, A3, A4, A5 × n) Among them, S d Indicates the size of the data description header, A1 indicates the number of bytes occupied by the compressed data identifier, A2 indicates the number of bytes occupied by the version number, A3 indicates the number of bytes occupied by the number of compressed data blocks, A4 indicates the number of bytes occupied by the compression method, A5 indicates the number of bytes occupied by the size of an original data block and its corresponding compressed data block size, and n indicates the number of compressed data blocks.

5. The radar-based data decoding method according to claim 1, characterized in that: Decompressing the radar-based data compression result specifically includes: Parsing the radar-based data compression result to obtain a data description header; Determine the size of the original radar base data according to the number of compressed data blocks in the data description header and the size of the original data block; Determine the decompression storage space according to the size of the original radar base data; Constructing a source address list in the compressed data area according to the size of each compressed data block in the data description header; constructing a target address list in the decompressed storage space according to the size of each original data block in the data description header; Based on the compression method in the data description header, multiple threads are used to simultaneously decompress all compressed data blocks, and the compressed data blocks are decompressed from the source address list to the target address list to obtain decompressed data blocks; According to the decompressed data blocks, decompressed data consistent with the binary-coded original radar base data is generated in the decompressed storage space.

6. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the radar-based data decoding method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the radar-based data decoding method according to any one of claims 1 to 5 is implemented.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the radar-based data decoding method according to any one of claims 1 to 5 is implemented.

Images