Method, device and storage medium for supporting multi-radar high-resolution networking

Abstract

The application discloses a kind of methods, equipment and storage medium supporting multi-radar high-resolution networking, it is related to radar data processing technical field.The method includes determining the spatial range of networking field;According to the spatial range and target resolution, define the grid structure of networking field data body;Allocation is used for high-performance computing computing resource;For each radar, pre-compute the mapping area of its detection range in grid structure;The original polar coordinate observation data of each radar is loaded into the computing resource in parallel;Based on the mapping area and original polar coordinate observation data, solve the networking value of each grid point in the grid structure layer by layer, generate networking result;The networking result is post-processed and stored, and the computing resource is released.The application solves the technical bottleneck of huge memory and display memory consumption fundamentally by solving from original polar coordinate observation data based on mapping area directly.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of radar data processing, and particularly relates to a method, device and storage medium for supporting multi-radar high-resolution networking based on hierarchical logic calculation. BACKGROUND

[0002] In recent years, radar detection technology in various fields has made great progress. Single system radar has been unable to meet the detection needs in complex scenarios. Therefore, networking technology that fuses data of multiple system radars (such as weather radars, phased array radars, etc.) has become the key to realizing large-scale, high-precision, and real-time situation awareness. By fusing the observation data of multiple radars into a unified and continuous two-dimensional or three-dimensional scene image, the monitoring and forecasting capabilities for meteorological targets, aircraft, and other targets are greatly improved.

[0003] In this technical evolution process, higher requirements are put forward for the accuracy, coverage range, and timeliness of data. Taking the meteorological industry as an example, modern fine weather forecasting, especially short-term nowcasting, urgently needs to realize rapid radar networking within a provincial or even regional range, with a kilometer-level or even higher resolution, and requires completion within an extremely short time (such as one minute) to timely capture rapidly evolving weather systems.

[0004] However, with the increase in the number of networked radars, the expansion of geographical range, and the improvement of spatial resolution, the amount of data generated during networking has shown explosive growth, posing extremely severe challenges to the memory capacity and computing performance of the computing architecture. Traditional networking algorithm processes usually follow two independent steps of "conversion first, then fusion": first, the original polar coordinate data (such as physical quantities represented by range, azimuth, and elevation, such as echo intensity dBZ and differential phase KDP) collected by each radar is converted into Cartesian coordinate system data with the same dimensions as the final target grid field (i.e., the networking field) through coordinate interpolation operations; then, all radar converted Cartesian coordinate data is fused and calculated to obtain the final networking product.

[0005] Taking a typical provincial 30-meter high-resolution three-dimensional networking scenario as an example, the networking field dimensions are (number of layers: 101, number of rows: 16000, number of columns: 19200). If int8 data type is used for storage, only the final networking result requires about 28.89 GB of memory or video memory space. Under the traditional method, if 80 radars participate in networking, the intermediate data in the Cartesian coordinate system of all 80 radars needs to be stored simultaneously before the fusion step. This will result in an astonishing memory occupancy of 80 x 28.89 GB = 2311.2 GB (about 2.3 TB). This data volume far exceeds the memory / video memory capacity of current mainstream single servers or single high-performance graphics cards, making this scheme impossible to execute on ordinary computing nodes.

[0006] Even if the memory problem is solved by means of technical means such as distributed computing or data block, the huge data carrying and repeated coordinate conversion calculation will cause serious I / O bottleneck and calculation delay, which makes the performance of networking processing decrease sharply and completely cannot meet the high timeliness requirement of "completing within one minute". Therefore, under the existing technical conditions, it is almost impossible to realize the radar networking application supporting more than 80 radars, 30-meter resolution, provincial range and completing within one minute.

[0007] In summary, there is an urgent need for a brand new radar data networking method to fundamentally solve the memory bottleneck problem caused by mass data intermediate storage and significantly improve the calculation efficiency to support larger scale and higher resolution real-time radar networking applications in the future. SUMMARY

[0008] In view of the above defects in the prior art, the purpose of the present application is to provide a method, device and storage medium supporting multi-radar high-resolution networking, to solve the technical bottleneck of huge memory consumption and extremely low calculation performance caused by the traditional 'conversion first and then fusion' method in the large-scale and high-resolution radar networking scene, so as to realize the near-real-time high-performance networking of more than 80 radars, 30-meter resolution and provincial range within one minute on a single machine or single card with limited memory capacity.

[0009] The present application solves the above technical problems by the following technical solutions: a method for supporting multi-radar high-resolution networking, comprising:

[0010] determining the spatial range of the networking field;

[0011] defining the grid structure of the networking field data body according to the spatial range and the preset target resolution;

[0012] allocating computing resources for high-performance computing;

[0013] pre-computing the mapping area of the detection range of each radar in the grid structure;

[0014] parallel loading the original polar coordinate observation data of each radar into the computing resources;

[0015] based on the mapping area and the original polar coordinate observation data, layer by layer solving the networking value of each grid point in the grid structure to generate the networking result;

[0016] post-processing and storing the networking result, and releasing the computing resources.

[0017] In this embodiment, by solving layer by layer based on the mapping area and the original polar coordinate observation data, on-demand calculation is realized, the step of generating a complete rectangular coordinate grid for each radar in advance in the traditional method is completely abandoned, which leads to memory explosion, the technical bottleneck of huge memory consumption is fundamentally solved, and the hardware demand is greatly reduced.

[0018] Due to the avoidance of generation, conversion and storage I / O of massive intermediate data, the computing burden is greatly reduced. Combined with "parallel loading" and "allocation of high-performance computing resources", the system can concentrate computing power in the effective area covered by the radar for fast solving, significantly improve the computing efficiency, and meet the near real-time business demand.

[0019] From "allocation of computing resources" to "parallel loading" to "post-processing and storage", the application constructs an efficient and smooth data processing pipeline. This integrated optimization design reduces the delay of data copying between different storage levels (such as disk, memory, and video memory), further ensuring high performance.

[0020] Further, the spatial range is the minimum outer package geographic area covering the detection range of all radars, which is calculated based on the latitude and longitude positions and effective detection radii of all radars participating in networking; the specific calculation process of the minimum outer package geographic area is:

[0021] Based on the center point latitude and longitude and the effective detection radius of each radar, the latitude and longitude boundary box of the detection range of each radar is calculated; the latitude and longitude boundary box includes the minimum longitude, maximum longitude, minimum latitude and maximum latitude;

[0022] The minimum outer package geographic area is synthesized by taking the minimum value of the minimum longitude, the minimum value of the minimum latitude, the maximum value of the maximum longitude and the maximum value of the maximum latitude in the latitude and longitude boundary box of all radars.

[0023] In this embodiment, the full-automatic calculation of the networking range is realized by calculating the minimum outer package geographic area, without manual intervention and custom range, avoiding the problems of incomplete radar data coverage (too small range) or waste of computing resources (too large range) caused by improper human setting, and ensuring the integrity and reliability of the networking result from the source.

[0024] By calculating the minimum outer package geographic area, a suitable grid is constructed in the horizontal dimension, which directly minimizes the number of rows and columns of the defined grid structure, thereby reducing unnecessary grid point calculation, reducing memory and video memory occupation, and improving I / O and storage efficiency.

[0025] Further, the grid structure of the defined networking field data body specifically includes:

[0026] calculating the row direction dimension of the grid according to the north-south latitude difference of the spatial range and the horizontal target resolution;

[0027] calculating the column direction dimension of the grid according to the east-west longitude difference of the spatial range and the central latitude and the horizontal target resolution;

[0028] calculating the layer direction dimension of the grid according to the vertical height range of the networking field and the vertical target resolution.

[0029] In the embodiment, by associating the north-south latitude difference and the east-west longitude difference (in combination with the central latitude) with the horizontal target resolution respectively, it is ensured that each pixel on the grid represents almost the same size of ground area in the real world within the entire networking area. This avoids the scale distortion caused by the curvature of the earth, so that the finally generated radar mosaic has accurate geometric meaning and can be perfectly superimposed with electronic maps and geographic information systems, providing reliable basis for accurate weather analysis and prediction.

[0030] The row, column and layer direction dimensions (i.e. the number of rows, columns and layers) calculated by the application are the minimum sufficient grid covering the target area, which is large enough to contain all necessary data and avoids defining a grid that is too large to generate a large number of invalid calculation units.

[0031] Further, the computing resources for high-performance computing are allocated, including:

[0032] creating a memory file system in the memory as a high-speed data cache area, applying a locked page memory for accelerating data exchange, creating a video memory cache area in the video memory, and creating a process pool and a thread pool.

[0033] Further, the raw polar coordinate observation data of each radar is loaded into the computing resources in parallel, specifically:

[0034] The raw polar coordinate observation data of each radar is constructed into independent loading tasks;

[0035] The loading tasks are allocated to multiple processes in the process pool to drive multiple processes to execute concurrently;

[0036] Each process reads and decompresses the raw polar coordinate observation data in the allocated task to the memory file system in parallel;

[0037] Through the applied locked page memory, the data located in the memory file system is asynchronously prefetched to the video memory cache area for GPU computing.

[0038] In the embodiment, the parallel reading and decompression of the radar data are realized by assigning independent loading tasks to the process pool for concurrent execution. This makes full use of the computing capacity of the multi-core CPU and the high-concurrency I / O capacity of the modern storage system, greatly shortens the data preparation time. The lock page memory allows the GPU to access directly through the DMA technology, avoids the data from being "locked" and copied by the operating system paging mechanism before transmission, realizes the transmission speed close to the bus bandwidth limit, enables the massive data to enter the GPU computing unit extremely fast, and avoids the GPU from idling due to waiting for data.

[0039] Further, for each radar, a mapping region of its detection range in the grid structure is pre-calculated, specifically including:

[0040] determining the latitude and longitude boundary box of the detection range based on the central point latitude and longitude and the effective detection radius of the radar;

[0041] converting the latitude and longitude boundary box into a floating-point index boundary in the grid structure;

[0042] performing an integer operation on the floating-point index boundary and restricting it within the actual index range of the grid structure to obtain the mapping region, the mapping region being defined by a minimum column index, a maximum column index, a minimum row index, and a maximum row index.

[0043] In the embodiment, the application generates a query table. In the subsequent processing of each grid point, complex spherical geometry calculations are no longer needed to determine whether the grid point is within the radar coverage range in real time, but only a few fast integer comparisons (determining whether the current point row and column number is within the pre-calculated index range) are needed, which greatly improves the calculation speed of the core networking algorithm.

[0044] Through the pre-calculated index range, it can be accurately determined which grid points need to be calculated and which grid points are outside the range of all radars. This enables the computing resources to be fully concentrated in the effective data area, completely avoiding futile interpolation and fusion operations in the data-free area. For a provincial large-scale networking, the invalid area may account for a considerable proportion, and this directly removes the huge calculation redundancy.

[0045] Further, based on the mapping region and the original polar coordinate observation data, the networking values of the grid points in the grid structure are solved layer by layer, specifically including: for each grid point in the current processing layer, the following steps are performed:

[0046] traversing all radars to determine whether the mapping region of the radar covers the current grid point;

[0047] If the current grid point is covered by at least one radar, for each radar covering the current grid point, the physical quantity value of the current grid point is calculated in real time by interpolation from the original polar coordinate observation data of the radar, and the networking value of the current grid point is determined according to all the physical quantity values.

[0048] If the current grid point is not covered by any radar, the current grid point is marked as an invalid value.

[0049] In the embodiment, it is no longer necessary to construct a complete rectangular coordinate data field in the memory for each radar, thereby completely avoiding the generation of massive intermediate data, completely abandoning the traditional step of pre-conversion, and fundamentally solving the problem of memory bottleneck; the selective calculation is realized through the judgment-calculation mode, the computing power is concentrated in the effective area covered by the radar data, the performance waste problem caused by invalid operation in the data-free area is completely avoided, and the computing efficiency is improved.

[0050] Real-time interpolation means that the most original observation data without intermediate interpolation loss is used for each calculation, thereby ensuring that the final networking result has higher fidelity and accuracy.

[0051] Further, the solving of the networking values of the grid points in the grid structure layer by layer comprises:

[0052] Before the solving, the original polar coordinate observation data of all the radars are preloaded into a first cache area of the video memory at one time, and a second cache area of the video memory is allocated for the layer networking result;

[0053] When solving any specific layer, the preloaded data in the first cache area is directly accessed, the networking values of the specific layer are calculated, and the networking values of the specific layer are written into the second cache area;

[0054] After completing the calculation and storage of all the networking values of the specific layer, the computing resources occupied by the solving of the specific layer are not released, and the solving of the next layer is circularly executed.

[0055] In the embodiment, it is no longer necessary to simultaneously accommodate the entire three-dimensional data body, but the huge three-dimensional calculation is decomposed into a series of manageable two-dimensional calculations, thereby solving the fundamental problem that the traditional method cannot be executed due to insufficient memory.

[0056] Based on the same concept, the application further provides an electronic device comprising a memory, a processor, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to realize the method for supporting multi-radar high-resolution networking as described above.

[0057] Based on the same concept, the application also provides a computer readable storage medium, which stores a computer program or instructions, and the computer program or instructions are executed by a processor to realize the method for supporting multi-radar high-resolution networking as described above.

[0058] Compared with the prior art, the application introduces a pre-computed mapping area, and solves layer by layer based on original polar coordinate observation data to construct a brand-new networking architecture, which fundamentally discards the traditional method of converting first and then fusing, thereby achieving the following beneficial effects:

[0059] By solving layer by layer, the memory requirement is reduced from having to accommodate the entire three-dimensional data body (TB level) to only processing single-layer data (GB level), so that it is possible to realize super-large-scale networking on a common server.

[0060] By solving based on the mapping area directly from the original polar coordinate observation data, the generation and conversion of massive intermediate data are avoided, the technical bottleneck of huge memory consumption is fundamentally solved, the computing resources are concentrated in the effective area, the processing time is greatly shortened, and the leap from offline batch processing to near-real-time processing is realized. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only one embodiment of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0062] Figure 1 is a flow chart of the method for supporting multi-radar high-resolution networking in the embodiment of the application;

[0063] Figure 2 is a schematic diagram of one embodiment of data storage and caching in the layered networking flow in the embodiment of the application;

[0064] Figure 3 is a schematic diagram of another embodiment of data storage and caching in the layered networking flow in the embodiment of the application;

[0065] Figure 4 is a combined echo (CR) effect diagram generated based on the grid structure (101, 16000, 19200) and 80 radar networking data in the embodiment of the application. DETAILED DESCRIPTION

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0068] Example 1

[0069] like Figure 1 As shown, the method for supporting high-resolution networking of multiple radars provided in this embodiment includes the following steps:

[0070] Step S1: Determine the spatial range of the network field.

[0071] Determine the spatial scope of the network, that is, determine the geographical area within which the network will be deployed. There are two methods for determining this:

[0072] The first method: Directly specify the latitude and longitude of the upper left and lower right corners of a rectangular area, and this rectangular area is the spatial range of the network field.

[0073] The second method: calculated based on the latitude and longitude positions of all participating radars and the effective detection radius of each radar. This embodiment prefers the second method. In a specific embodiment of the present invention, calculating the spatial range of the network field based on the latitude and longitude positions of all participating radars and the effective detection radius of each radar specifically includes:

[0074] Step S1.1: Based on the center point latitude and longitude of each radar and the effective detection radius R, calculate the latitude and longitude bounding box of the detection range of each radar; the latitude and longitude bounding box includes the minimum longitude, maximum longitude, minimum latitude and maximum latitude.

[0075] For each radar, the latitude and longitude coordinates of the point reached after moving R from its center point in four directions (east, west, north, and south) constitute the latitude and longitude boundary of the radar's detection range.

[0076] Step S1.2: By taking the minimum value of the minimum longitude, the minimum value of the minimum latitude, the maximum value of the maximum longitude, and the maximum value of the maximum latitude in the latitude and longitude bounding boxes of all radars, the minimum bounding geographical area is synthesized, which is the spatial range of the network field.

[0077] For each radar, its latitude-longitude bounding box is defined by a pair of longitude extremes and a pair of latitude extremes. Among them, the west side boundary corresponds to the minimum longitude, and the east side boundary corresponds to the maximum longitude; the south side boundary corresponds to the minimum latitude, and the north side boundary corresponds to the maximum latitude.

[0078] Then, the minimum value is selected from the minimum longitude of the latitude-longitude bounding box of all radars, the minimum value is selected from the minimum latitude, the maximum value is selected from the maximum longitude, and the maximum value is selected from the maximum latitude, thereby forming a minimum outer-geographic area.

[0079] Through the second mode, the application can automatically, efficiently and accurately determine a minimum spatial range which has no redundancy and can ensure complete coverage of all radar data, thereby laying a foundation for subsequent grid division and efficient calculation.

[0080] Step S2: Defining the grid structure of the networking field data body according to the spatial range and the preset target resolution.

[0081] The essence of step S2 is geographic coordinate discretization, which needs to convert a continuous geographic area (described by latitude and longitude) into a discrete network composed of countless small cells, and the size of each small cell is determined by the target resolution.

[0082] In one specific embodiment of the application, the grid structure of the networking field data body is defined, specifically including:

[0083] Step S2.1: Calculating the row direction dimension of the grid according to the north-south latitude difference of the spatial range and the horizontal target resolution.

[0084] Let the lower left latitude and longitude of the spatial range be (lon_min, lat_min), and the upper right latitude and longitude be (lon_max, lat_max), and the horizontal target resolution be resolution_h (for example: 30 meters). Since the latitude and longitude are angle units, and the horizontal target resolution is distance unit, the angle unit of latitude and longitude needs to be converted into distance unit first, 1 degree of latitude ≈ 111,000 meters. Therefore, the north-south latitude difference of the spatial range is height_m = (lat_max - lat_min) x 111000, and the row direction dimension (i.e. the number of rows) is n_rows = height_m / resolution_h.

[0085] Step S2.2: Calculating the column direction dimension of the grid according to the east-west longitude difference of the spatial range, the central latitude, and the horizontal target resolution.

[0086] The formula for the distance of 1 degree longitude varies with latitude is 111000 x cos(latitude x pi / 180). To represent the whole spatial range fairly, the center latitude of the spatial range is taken for the calculation. The center latitude is lat_center = (lat_min + lat_max) / 2, the east-west longitude difference of the spatial range is width_m = (lon_max - lon_min) x 111000 x cos(lat_center x pi / 180), and the column dimension (i.e. the number of columns) is n_cols = width_m / resolution_h.

[0087] To ensure that the whole spatial range is covered, if the calculated row and column dimensions are fractional, they should be rounded up to avoid losing a side due to the fractional part.

[0088] Step S2.3: Calculate the layer dimension of the grid according to the vertical height range of the networking field and the vertical target resolution.

[0089] Let the lowest height in the vertical height range be altitude_min (e.g. 0 meters, i.e. the ground), the highest height be altitude_max, and the vertical target resolution be resolution_v. Since the height is linear distance, the height difference in the vertical direction is depth_m = altitude_max - altitude_min, and the layer dimension (i.e. the number of layers) is n_levels = depth_m / resolution_v. Again, the calculated layer dimension should be rounded up.

[0090] Step S3: Allocate computing resources for high-performance computing.

[0091] The allocation of computing resources for high-performance computing aims to build a low-latency, high-throughput data processing environment, including:

[0092] Step S3.1, configure computing and storage resource parameters: Before the program starts or during runtime, the following key parameters are pre-set through configuration files or command line parameters:

[0093] Parallel computing scale: Set the number of processes and threads for data loading and post-processing;

[0094] Cache resource configuration: Set the path and cache size of the data cache directory.

[0095] Step S3.2, initialize the high-performance computing environment: During the program initialization stage, the following operations are performed according to the above configuration:

[0096] Memory cache area is established: a memory area consistent with the configured cache size is opened in the operating system memory, and is mounted as tmpfs (temporary file system) and mapped to the data cache directory specified in the configuration. This operation builds a virtual file path in memory as a high-speed data transfer station. All subsequent data reading, decompression and intermediate storage are completed in this directory. Since data does not need to be written to the physical disk, disk I / O delay is completely avoided, and extremely high data throughput close to the theoretical bandwidth of memory is obtained.

[0097] Lock page memory is applied: using the cupy.cuda.PinnedMemoryPool library function, a fixed physical memory that will not be swapped to disk by the operating system, i.e. lock page memory, is applied.

[0098] The lock page memory is used to enable direct memory access (DMA) mechanism when transmitting data between the host memory and the GPU memory, eliminating the bottleneck of data copying through the CPU before transmission, thereby greatly accelerating the data flow between the CPU and the GPU.

[0099] Global memory cache area is created: one or more global memory spaces are opened in the global memory of the GPU as global memory cache areas for storing data to be calculated by the GPU.

[0100] Process pool and thread pool are created: according to the pre-configured number of processes and threads, a process pool and a thread pool are pre-created. The process pool is mainly used for parallel execution of radar data loading and decompression and other I / O intensive tasks, and the thread pool is mainly used for parallel execution of network result post-processing and storage and other computationally intensive tasks.

[0101] This pooling technique avoids the system overhead of frequently creating and destroying processes / threads during task execution, realizes the reuse of computing resources, and ensures stable and efficient program execution.

[0102] Through the above steps, a high-performance computing environment from memory, global memory to parallel computing power is built, laying a solid foundation for subsequent high-speed data loading and processing.

[0103] Step S4: For each radar, the mapping area of its detection range in the grid structure is pre-calculated.

[0104] Step S4 aims to establish an index for each radar to quickly determine its precise influence range in the final network grid. In one specific embodiment of the present application, for each radar, the mapping area of its detection range in the grid structure is pre-calculated, specifically including:

[0105] Step S4.1: Based on the radar's center point longitude and latitude and effective detection radius, determine the longitude and latitude boundary box of its detection range.

[0106] In this embodiment, the longitude and latitude boundary box of the radar's detection range can be calculated by using spherical geometry formula (such as Haversine formula) or plane approximation method.

[0107] Step S4.2: Convert the longitude and latitude boundary box into floating-point index boundary in the grid structure.

[0108] Map the longitude and latitude boundary box obtained in step S4.1 into the network coordinate system of the networking field to obtain the index boundary in floating-point number form (i.e., floating-point index boundary).

[0109] Step S4.3: Perform rounding operation on the floating-point index boundary and constrain it within the actual index range of the grid structure to obtain the mapping area, which is defined by the minimum column index, the maximum column index, the minimum row index, and the maximum row index.

[0110] To ensure complete coverage of the radar range, perform rounding operation on the floating-point index boundary to convert it into regular integer index. Specifically, round down the minimum column index and the minimum row index, and round up the maximum column index and the maximum row index. Constrain the rounded index within the actual index range of the grid, i.e., the index is located within the effective grid range.

[0111] Finally, the mapping area of the radar is defined as the rectangular area bounded by the rounded and constrained minimum column index, maximum column index, minimum row index, and maximum row index. This area clearly indicates the grid row and column range that needs to access the radar data in subsequent networking calculations.

[0112] Through the above steps, an accurate grid index query table is generated for each radar, laying a solid foundation for subsequent implementation of "on-demand computing" and performance optimization.

[0113] Step S5: Load the original polar coordinate observation data of each radar into the computing resource in parallel.

[0114] The core goal of step S5 is to build a high-throughput and low-latency data pipeline from the storage medium to the GPU memory. In one specific embodiment of the present application, the original polar coordinate observation data of each radar is loaded into the computing resource in parallel, specifically:

[0115] Step S5.1: Construct the original polar coordinate observation data of each radar into independent loading tasks.

[0116] An independent loading task is constructed for each raw data file of each radar (usually the polar coordinate observation data of a complete body scan of a radar), which contains the path of the data file, the decompression parameters, the target cache location and other meta information. This operation ensures that the data loading activities of all radars can be carried out independently and without interference.

[0117] Step S5.2: Distribute the loading tasks to multiple processes in the process pool to drive multiple processes to execute concurrently.

[0118] The set of loading tasks generated by the above construction is submitted to the pre-created process pool as a whole. The scheduler of the process pool dynamically distributes these tasks to multiple working processes in the process pool according to the system load and the number of tasks. For example, if there are 80 radars and 40 processes, the scheduler will roughly distribute the tasks of every 2 radars to a process, thereby driving multiple processes to execute concurrently and fully utilizing the computing power of the multi-core CPU.

[0119] Step S5.3: Each process reads and decompresses the raw polar coordinate observation data in the assigned task to the memory file system in parallel.

[0120] After receiving the assigned task, each working process independently accesses the storage system to read the raw polar coordinate observation data file specified in the assigned task into the memory of the operating system. If the raw data is in compressed format (such as BZIP2 or GZIP), the process directly decompresses it in memory to restore the standard radar base data format.

[0121] After decompression, the process writes the decompressed data to the initialized memory file system (i.e. tmpfs in step S3.2). Since the memory file system is entirely in memory, the writing operation is extremely fast, avoiding the bottleneck of traditional physical disk I / O. At this point, the raw data of all radars has been converted to a format that can be used for computation and stored in a high-speed memory cache area.

[0122] Step S5.4: Asynchronously prefetch the data in the memory file system to the video memory cache area through the requested locked page memory for GPU computation.

[0123] To enable efficient computation by the GPU, the data needs to be further transferred to the video memory:

[0124] An efficient data transmission channel from main memory to video memory is established through the requested locked page memory.

[0125] An independent control thread or process is responsible for monitoring the data readiness status in the memory file system and initiating an asynchronous data prefetch operation. Specifically, it copies data from the tmpfs to the lock page memory, and then efficiently and asynchronously transfers the data from the lock page memory to the video memory cache area through the direct memory access (DMA) technology. Asynchronous means that the transmission process is dominated by the DMA engine of the GPU, without the continuous participation of the CPU, and overlaps with other computing tasks that the GPU may be performing, thereby hiding the transmission delay.

[0126] Through the above steps, a highly parallelized and pipelined data loading process is completed, and the dispersed original radar data is efficiently delivered to the GPU computing unit, fully preparing for the subsequent real-time networking calculation. This step ensures that the data loading stage will not become a performance bottleneck of the entire real-time processing process.

[0127] Step S6: Based on the mapping area and the original polar coordinate observation data, the networking value of each grid point in the grid structure is solved layer by layer to generate the networking result.

[0128] In one specific embodiment of the present application, based on the mapping area and the original polar coordinate observation data, the networking value of each grid point in the grid structure is solved layer by layer, specifically including:

[0129] Step S6.1: Before solving, all original polar coordinate observation data of the radars are preloaded to the first cache area of the video memory at one time.

[0130] Before starting the layer-by-layer calculation, the data preloading operation is first performed. Specifically, all original polar coordinate observation data collected by the radars are preloaded to the first cache area of the video memory at one time through the PCIe bus. The video memory is logically divided into two cache areas: the first cache area is used to store all preloaded original polar coordinate observation data of the radars; and the second cache area is used to temporarily store the networking result of the current layer during the solving process.

[0131] Step S6.2: For any specific layer, without reloading the original radar data, the preloaded data in the first cache area is directly accessed to perform calculation.

[0132] In the GPU-accelerated implementation of the calculation, a parallel thread is started for each grid point of the specific layer, and the following operations are synchronously performed:

[0133] For the current grid point P(i, j) (where i is the row index and j is the column index), all radars are traversed to determine whether the mapping area of each radar covers the current grid point P(i, j), i.e., whether the current grid point P(i, j) is located in the mapping area of a certain radar.

[0134] If the current grid point P(i, j) is covered by at least one radar, for each radar covering the current grid point P(i, j), the corresponding position (distance, azimuth) of the current grid point P(i, j) in the radar polar coordinate system is calculated in real time according to the geographic coordinates of the current grid point P(i, j); the original polar coordinate observation data of the radar in the first cache area of the display memory is directly accessed, and the physical quantity value V of the current grid point P(i, j) in the radar view is calculated by an interpolation algorithm (such as bilinear interpolation) based on the calculated position. k (where k is the radar number).

[0135] After obtaining the physical quantity values {V1, V2,..., V n} of all related radars, the networking value V final of the current grid point P(i, j) is calculated according to a preset fusion algorithm (such as taking the maximum value, the average value, or the weighted average), and is written into the second cache area.

[0136] If the current grid point is not covered by any radar, no interpolation calculation is needed, and the current grid point in the second cache area is directly marked as an invalid value.

[0137] Step S6.3: After completing the calculation of all the networking values of the specific layer, the networking result of the layer in the second cache area is stored in the system memory. In this process, the calculation resources occupied by the radar original data in the first cache area and the second cache area are not released, and their states are maintained for the next layer to be used cyclically.

[0138] In this step, whenever the calculation of a layer is completed, the networking result of the layer is copied from the display memory to the memory, and the post-processing process is notified. Subsequently, the display memory resources occupied by the layer are not released, so as to start the cache space reuse for the next layer calculation.

[0139] Step S6.4: Steps S6.2 to S6.3 are repeatedly executed until the calculation of all layers in the grid structure is completed.

[0140] Step S7: The networking result is post-processed and stored, and the calculation resources are released.

[0141] The single-layer networking result transmitted from step S6 is received, and is post-processed in parallel (such as data compression, format conversion, invalid value filtering, etc.) by using a thread pool, and the finally processed data is written into a persistent storage system (such as a disk or a database).

[0142] According to the use requirements of subsequent data, the memory cache area adopts two data organization modes:

[0143] Layer-priority mode: When the application needs to quickly access the 2D networking results of a specific layer (e.g., a certain height layer), the system will directly write the layer-by-layer 2D data into the persistent storage system (e.g., disk or database), as shown in Figure 2 . Figure 2 The process of storing 3D networking data as multiple 2D data by layer level under the layered processing logic is demonstrated, including the data flow relationship between storage, memory cache, locked page memory, video memory cache, and reusable single-layer networking cache.

[0144] Block-priority mode: When the application needs to quickly access a local area in 3D space, the system will divide the networking results into smaller data blocks in 3D, temporarily store them in the block-level cache area, and finally write them into the persistent storage system (e.g., disk or database), as shown in Figure 3 . Figure 3 The process of storing 3D networking data as multiple 3D data by block division under the layered processing logic is demonstrated, including the data organization relationship between storage, memory cache, locked page memory, video memory cache, and reusable single-layer networking cache.

[0145] When all layer networking calculations and post-processing are completed, the system releases all allocated computing resources, including: destroying process pool and thread pool, releasing video memory cache area, releasing locked page memory, and unloading and releasing memory file system (tmpfs).

[0146] Assuming the 3D dimensions of the networking field data volume are (101, 16000, 19200), representing a total of 101 layers, 16000 rows, and 19200 columns. Take the calculation of a specific grid point in the 10th layer as an example.

[0147] The current processing layer is the 10th layer, which is a two-dimensional grid with a data size of (16000, 19200). The goal of this embodiment is to calculate the final networking value of the grid point with coordinates (y=10000, x=15000) in this layer.

[0148] Traverse all radar mapping regions to determine whether the plane coordinates (x=15000, y=10000) of the grid point [10, 10000, 15000] fall within the mapping region of a radar. Assuming that through judgment, it is determined that the grid point is covered by radar A, radar B, and radar C.

[0149] From the original polar coordinate observation data of radar A, radar B, and radar C loaded into the cache, the physical quantity values (e.g., reflectivity factor dBZ) corresponding to the grid point in each radar data are obtained through real-time coordinate inversion and interpolation calculation (e.g., bilinear interpolation).

[0150] Assume the calculated values ​​are: V A =35.2 dBZ, V B = 41.5 dBZ, V C = 38.7 dBZ. According to the preset maximum value fusion algorithm, these three values ​​are compared, and the maximum value is taken as the final network result for that grid point. That is: V final =max(35.2,41.5,38.7)=41.5dBZ. Finally, the value 41.5 dBZ was filled into the (10000, 15000) position in the 10th layer networking result data.

[0151] The data organization in the memory cache corresponds to the specific storage model:

[0152] If adopted Figure 2 In the layer-first mode shown, 101 cache spaces with dimensions (16000, 19200) are allocated in memory. After the networking results of each layer are calculated in the GPU, they can be directly written to their corresponding layer cache space via DMA, making the process highly efficient.

[0153] If adopted Figure 3 The block-first mode shown takes a block dimension of (101, 400, 480) as an example, creating 1600 cache blocks of 40×40 in memory. In this case, the networking result of each layer needs to be split into 40×40 small blocks, and then written to the corresponding layer position in each cache block via DMA.

[0154] Furthermore, during the layer-by-layer calculation of network values, combined echo (CR) products can be generated simultaneously. The system maintains a buffer with dimensions (16000, 19200) (initially with invalid values). After each layer is processed, the system performs a point-by-point maximum value calculation: CR_current = max(CR_previous, Layer_result). Here, CR_current represents the combined echo data obtained up to the current layer; CR_previous represents the combined echo data before processing the current layer, i.e., the previous state of the combined echo data; and Layer_result represents the network value of all grid points in the current layer. When all layers are processed, the final CR result is stored in this buffer.

[0155] Figure 4The CR effect obtained thereby is shown, with the horizontal and vertical coordinates being grid points, the color band representing the dBZ value, and the CR shape being the data dimension of the combined echo. This CR calculation process embodies the high expansibility advantage of the hierarchical networking logic: it skillfully decomposes the high memory consumption Z-axis maximum value operation in the traditional method, which needs to be performed on the complete three-dimensional data body, into lightweight operations that can be performed iteratively with layers, thereby significantly reducing the peak memory occupancy.

[0156] Embodiment Two

[0157] The embodiment of the present application also provides an electronic device, comprising a memory, a processor and a computer program or instructions stored in the memory, the processor executes the computer program or instructions to realize the method for supporting multi-radar high-resolution networking in the embodiment one of the present application.

[0158] Although not shown, the electronic device comprises a processor, which can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage part into a random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special-purpose coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), etc. In the RAM, various programs and data required for device operation are also stored. The processor, ROM and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0159] The above processor and memory are used together to execute programs / instructions stored in the memory, which can realize the methods, steps or functions described in the above embodiments when executed by a computer.

[0160] Although not shown, the embodiment of the present application also provides a computer readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, realizes the method for supporting multi-radar high-resolution networking in the embodiment one of the present application.

[0161] Read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store information which can be accessed by a computing device. According to the definition used herein, computer readable medium does not include transitory media, such as modulated data signals and carrier waves.

[0162] The above disclosure is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or modifications within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for supporting high-resolution networking of multiple radars, characterized in that, The method includes: Determine the spatial scope of the network field; Define the grid structure of the network field data volume based on the spatial range and the preset target resolution; Allocate computing resources for high-performance computing; For each radar, pre-calculate the mapping area of ​​its detection range in the grid structure; The raw polar coordinate observation data of each radar are loaded into the computing resources in parallel; Based on the mapped region and the original polar coordinate observation data, the network values ​​of each grid point in the grid structure are solved layer by layer to generate the network results, which specifically include: Before starting the layer-by-layer solution, all the original polar coordinate observation data of the radar are preloaded into the first cache of the video memory at once, and the second cache of the video memory is allocated for the layer networking results. When solving any layer of the mesh structure, directly access the preloaded data in the first buffer, and for each mesh point in the current processing layer, perform the following steps: Traverse all radars and determine whether their mapping area covers the current grid point; If the current grid point is covered by at least one radar, then for each radar covering the current grid point, the physical quantity value of the current grid point is interpolated in real time from its original polar coordinate observation data, the network value of the current grid point is determined based on all physical quantity values, and the network value is written into the second buffer. If the current grid point is not covered by any radar, then the current grid point is marked as invalid; after completing the calculation and storage of all network values ​​of the current processing layer, the computing resources occupied by the current processing layer during the solution are not released, and the solution of the next layer is executed in a loop until the calculation of all layers in the network structure is completed; The networking results are post-processed and stored, and the computing resources are released.

2. The method for supporting high-resolution networking of multiple radars according to claim 1, characterized in that, The spatial range is the smallest outer geographical area that can cover the detection range of all radars, calculated based on the latitude and longitude positions and effective detection radii of all participating radars. The specific calculation process for the smallest outer geographical area is as follows: Based on the latitude and longitude of the center point of each radar and its effective detection radius, the latitude and longitude bounding box of the detection range of each radar is calculated; the latitude and longitude bounding box includes the minimum longitude, maximum longitude, minimum latitude, and maximum latitude; The minimum bounding geographic region is synthesized by taking the minimum value of the minimum longitude, the minimum value of the minimum latitude, the maximum value of the maximum longitude, and the maximum value of the maximum latitude from the latitude and longitude bounding boxes of all radars.

3. The method for supporting high-resolution networking of multiple radars according to claim 1, characterized in that, The defined mesh structure of the network field data volume specifically includes: The row direction dimension of the grid is calculated based on the north-south latitude difference and the horizontal target resolution of the spatial range. The column direction dimension of the grid is calculated based on the east-west longitude difference and center latitude of the spatial range and the horizontal target resolution. Calculate the layer orientation dimension of the mesh based on the vertical height range of the mesh field and the vertical target resolution.

4. The method for supporting high-resolution networking of multiple radars according to claim 1, characterized in that, The allocation of computing resources for high-performance computing includes: Create a memory file system in memory as a high-speed data cache, allocate paged memory to accelerate data exchange, create a video memory cache in video memory, and create a process pool and a thread pool.

5. The method for supporting high-resolution networking of multiple radars according to claim 4, characterized in that, The process of loading the raw polar coordinate observation data of each radar into the computing resources in parallel specifically involves: The raw polar coordinate observation data of each radar are constructed into independent loading tasks; The loading task is assigned to multiple processes in the process pool to drive the concurrent execution of multiple processes; Each process reads and decompresses the raw polar coordinate observation data from its assigned task to the memory file system in parallel; The data located in the memory file system is asynchronously prefetched to the video memory cache through the requested page lock memory for GPU computing.

6. The method for supporting high-resolution networking of multiple radars according to claim 1, characterized in that, For each radar, its detection range is pre-calculated within the mapped area of ​​the grid structure, specifically including: Based on the latitude and longitude of the radar's center point and its effective detection radius, determine the latitude and longitude bounding box of its detection range; Convert the latitude and longitude bounding box into a floating-point indexed bounding box in the grid structure; The floating-point index boundary is rounded and constrained within the actual index range of the grid structure to obtain the mapping region, which is defined by the minimum column index, the maximum column index, the minimum row index, and the maximum row index.

7. An electronic device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The processor executes the computer program or instructions to implement the method for supporting high-resolution networking of multiple radars as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by the processor, they implement the method for supporting high-resolution networking of multiple radars as described in any one of claims 1 to 6.

Images