Dynamic distributed short-weather dispatching method and system for short-weather precipitation prediction

Abstract

The application relates to a dynamic distributed short-term scheduling method and system for short-term precipitation prediction, which comprises the following steps: inquiring whether new radar echo data is received in a preset storage folder; if the new radar echo data is received in the storage folder, the new radar echo data is segmented, and a plurality of calculation units are obtained after the segmentation; the segmented calculation units are stored, and the stored calculation units are dynamically allocated to each precipitation prediction processing machine; precipitation prediction is performed on the calculation units based on a preset space-time prediction model, and precipitation prediction results of the calculation units are respectively generated; the precipitation prediction results are combined and precipitation prediction data is generated. The application realizes efficient scheduling of multiple machines for dynamic distributed short-term scheduling.

Description

Technical Field

[0001] This application relates to the field of weather data processing technology, and in particular to a dynamic distributed short-term scheduling method and system for short-term precipitation forecasting. Background Technology

[0002] Short-term weather forecasts, also known as short-term weather predictions, refer to weather forecasts with a lead time of 24-72 hours. Forecasting short-term weather, especially convective weather systems, is challenging due to their complex mechanisms and small spatial and temporal scales, often resulting in vastly different weather conditions within short distances. This makes forecasting extremely difficult, and short-term forecasting remains a global challenge.

[0003] With the increasing necessity of short-term forecasting, short-term precipitation prediction has become a research focus in the field of weather forecasting, and many prediction models based on neural network algorithms have emerged. However, regardless of the algorithm model, a large number of convolution operations are required. The characteristic of short-term precipitation prediction is that it generates prediction results in a short period of time, which requires powerful computing power. Increasing computing power will correspondingly increase costs.

[0004] As is well known, powerful computing power is required for short-term precipitation forecasting. However, increasing the computing power of a single machine is costly and difficult. Therefore, a dynamic distributed short-term scheduling method for short-term precipitation forecasting is needed to dynamically schedule multiple machines. Summary of the Invention

[0005] Therefore, it is necessary to provide a dynamic distributed short-term scheduling method and system for short-term precipitation forecasting that avoids the high cost and difficulty of increasing the computing power of a single machine to meet the computational needs of short-term weather forecasting, and achieves efficient scheduling of multiple machines.

[0006] The technical solution of this invention is as follows:

[0007] A dynamic distributed short-term scheduling method for short-term precipitation forecasting, the method comprising:

[0008] Step S100: Check if new radar echo data has been received in the preset storage folder;

[0009] Step S200: If the new radar echo data is received in the storage folder, the new radar echo data is segmented, and multiple computing units are obtained after the segmentation is completed;

[0010] Step S300: Store the segmented computing units and dynamically allocate the stored computing units to each precipitation prediction processing machine. After allocation, each precipitation prediction processing machine is allocated the same amount of computing unit data.

[0011] Step S400: Based on the preset spatiotemporal prediction model, precipitation prediction is performed on the calculation units respectively, and precipitation prediction results of each calculation unit are generated. The precipitation prediction results are then merged to generate precipitation prediction data.

[0012] Furthermore, step S300 involves storing the segmented computing units and dynamically allocating the stored computing units to each precipitation prediction processing machine. After allocation, each precipitation prediction processing machine receives the same amount of data from its allocated computing units. Specifically, this includes:

[0013] Step S310: Store the segmented data of each computing unit into a preset segmentation matrix;

[0014] Step S320: Obtain the amount of data contained in each of the stored computing units;

[0015] Step S330: Obtain the pre-set number of precipitation prediction processing machines;

[0016] Step S340: Allocate each computing unit to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, wherein, after the allocation is completed, the amount of data allocated to each computing unit by each precipitation prediction processing machine is the same.

[0017] Furthermore, step S340: allocating each computing unit to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, specifically includes:

[0018] Step S341: Randomly select n initial computing units from the set data of computing units and assign them to n precipitation prediction processing machines, wherein the set data of computing units contains all computing units;

[0019] Step S342: Sort the n precipitation prediction and processing machines in ascending order of the amount of data they possess;

[0020] Step S343: Randomly select n computing units from the set of computing units data and assign them to n precipitation prediction processing machines;

[0021] Step S344: Repeat steps S341-S343 until the amount of data allocated to each of the precipitation prediction processing machines is the same, at which point the allocation ends.

[0022] Furthermore, step S400: Based on a preset spatiotemporal prediction model, precipitation prediction is performed on each of the computing units, and precipitation prediction results are generated for each computing unit. The precipitation prediction results are then merged to generate precipitation prediction data, specifically including:

[0023] Step S410: After the calculation unit is allocated to each precipitation prediction processing machine, a prediction start command is sent to each precipitation prediction processing machine;

[0024] Step S420: According to the prediction start command, control each precipitation prediction processing machine to perform precipitation prediction on the calculation unit based on the spatiotemporal prediction model, and generate precipitation prediction results for each calculation unit respectively;

[0025] Step S430: Check whether all the precipitation prediction processing machines have completed their predictions;

[0026] Step S440: If all the precipitation prediction processing machines have completed their predictions, then the precipitation prediction results are merged and precipitation prediction data is generated.

[0027] Furthermore, step S200: If the new radar echo data is received in the storage folder, the new radar echo data is segmented, and multiple computing units are obtained after the segmentation; specifically including:

[0028] Step S210: If the new radar echo data is received in the storage folder, determine whether the data coverage area of ​​the new radar echo data is an integer multiple of the preset cutting area;

[0029] Step S220: If it is determined that the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is filled.

[0030] Step S230: Perform data segmentation on the newly filled radar echo data, and obtain multiple computing units after the segmentation is completed.

[0031] Furthermore, in step S220: if it is determined that the size of the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is being filled, and the relationship after filling is as follows:

[0032]

[0033] Where cwn is the number of horizontal segments, chn is the number of vertical segments, roundup is the round-up operation, ow is the width of the original data coverage area, cw is the width of the prediction area, oh is the height of the original data coverage area, ch is the height of the prediction area, h is the height after expansion, and w is the width after expansion.

[0034] Furthermore, a dynamic distributed short-term scheduling system for short-term precipitation forecasting, the system comprising:

[0035] The file receiving module is used to query whether new radar echo data has been received in a preset storage folder;

[0036] The segmentation module is used to segment the new radar echo data if the new radar echo data is received in the storage folder, and to obtain multiple computing units after the segmentation is completed.

[0037] The segmentation module is used to store the segmented computing units and dynamically allocate the stored computing units to each precipitation prediction processing machine. After the allocation is completed, each precipitation prediction processing machine is allocated the same amount of computing unit data.

[0038] The merging module is used to perform precipitation prediction on the calculation units based on a preset spatiotemporal prediction model, generate precipitation prediction results for each calculation unit, and merge the precipitation prediction results to generate precipitation prediction data.

[0039] Furthermore, the segmentation module is also used for:

[0040] The segmented data of each computing unit is stored in a preset segmentation matrix; the amount of data contained in each of the stored computing units is obtained; the number of precipitation prediction processing machines is obtained in advance; each computing unit is allocated to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, wherein, after the allocation is completed, the amount of computing unit data allocated to each precipitation prediction processing machine is the same.

[0041] The merging module is further configured to: after the computing units are allocated to each precipitation prediction processing machine, send a prediction start command to each precipitation prediction processing machine; control each precipitation prediction processing machine to perform precipitation prediction on the computing units based on the spatiotemporal prediction model according to the prediction start command, and generate precipitation prediction results for each computing unit respectively; detect whether all precipitation prediction processing machines have completed the prediction; if all precipitation prediction processing machines have completed the prediction, merge the precipitation prediction results and generate precipitation prediction data.

[0042] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps described above in the dynamic distributed short-term scheduling method for short-term precipitation forecasting.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described above in the dynamic distributed short-term scheduling method for short-term precipitation forecasting.

[0044] The technical effects achieved by this invention are as follows:

[0045] The aforementioned dynamic distributed short-term scheduling method and system for short-term precipitation forecasting sequentially queries a preset storage folder to see if new radar echo data has been received. If the new radar echo data is found in the storage folder, it is segmented to obtain multiple computing units. The segmented computing units are stored, and then dynamically allocated to precipitation forecasting processing machines. After allocation, each precipitation forecasting processing machine receives the same amount of data from its assigned computing units. Precipitation forecasts are performed on each computing unit based on a preset spatiotemporal prediction model, generating precipitation forecast results for each unit. These results are then merged to generate precipitation forecast data. This avoids the high cost and difficulty associated with increasing the computing power of a single machine to meet the computational needs of short-term weather forecasting, thus achieving efficient scheduling of multiple machines for dynamic distributed short-term forecasting. Attached Figure Description

[0046] Figure 1 This is a block diagram of a dynamic distributed short-term scheduling system for short-term precipitation forecasting in one embodiment.

[0047] Figure 2 This is a flowchart illustrating a dynamic distributed short-term scheduling method for short-term precipitation forecasting in one embodiment.

[0048] Figure 3 This is a schematic diagram illustrating the filling of computational units in a dynamic distributed short-term scheduling method for short-term precipitation forecasting in one embodiment.

[0049] Figure 4 This is a network structure diagram of the spatiotemporal prediction model used in a dynamic distributed short-term scheduling method for short-term precipitation prediction in one embodiment.

[0050] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] In one embodiment, an application scenario for a dynamic distributed short-term scheduling method for short-term precipitation forecasting is provided. This application scenario includes a smart terminal that queries a preset storage folder to see if new radar echo data has been received. If the new radar echo data is found in the storage folder, it is segmented to obtain multiple computing units. The segmented computing units are stored, and the stored computing units are dynamically allocated to precipitation forecasting processing machines. After allocation, each precipitation forecasting processing machine receives the same amount of data from its assigned computing units. Precipitation forecasts are performed on each computing unit based on a preset spatiotemporal prediction model, and precipitation forecast results are generated for each computing unit. The precipitation forecast results are then merged to generate precipitation forecast data.

[0053] The smart terminal may be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0054] In one embodiment, a dynamic distributed short-term scheduling system for short-term precipitation forecasting is provided, such as... Figure 1 As shown, the system includes:

[0055] The file receiving module is used to query whether new radar echo data has been received in a preset storage folder;

[0056] The segmentation module is used to segment the new radar echo data if the new radar echo data is received in the storage folder, and to obtain multiple computing units after the segmentation is completed.

[0057] The segmentation module is used to store the segmented computing units and dynamically allocate the stored computing units to each precipitation prediction processing machine. After the allocation is completed, each precipitation prediction processing machine is allocated the same amount of computing unit data.

[0058] The merging module is used to perform precipitation prediction on the calculation units based on a preset spatiotemporal prediction model, generate precipitation prediction results for each calculation unit, and merge the precipitation prediction results to generate precipitation prediction data.

[0059] The system also includes a monitoring module, which is responsible for ensuring the consistency of the precipitation prediction and processing by each machine.

[0060] The storage folder is used to store newly arrived radar echo data files. The segmentation matrix stores the segmented results, serving as input for the prediction task. The prediction matrix stores the predicted results. The state array stores the execution status of each precipitation prediction processing machine for use by the monitoring module.

[0061] In one embodiment, such as Figure 2 As shown, a dynamic distributed short-term scheduling method for short-term precipitation forecasting is provided, the method comprising:

[0062] Step S100: Check if new radar echo data has been received in the preset storage folder;

[0063] In this step, radar data is pushed to the storage folder every 6 minutes via FTP. Once the file receiving module receives new data, it notifies the scheduling system of the new file, and the scheduling system then activates the segmentation module to split the file data. FTP stands for File Transfer Protocol.

[0064] Step S200: If the new radar echo data is received in the storage folder, the new radar echo data is segmented, and multiple computing units are obtained after the segmentation is completed;

[0065] Step S300: Store the segmented computing units and dynamically allocate the stored computing units to each precipitation prediction processing machine. After allocation, each precipitation prediction processing machine is allocated the same amount of computing unit data.

[0066] Step S400: Based on the preset spatiotemporal prediction model, precipitation prediction is performed on the calculation units respectively, and precipitation prediction results of each calculation unit are generated. The precipitation prediction results are then merged to generate precipitation prediction data.

[0067] Furthermore, the present invention

[0068] In one embodiment, step S200: If the new radar echo data is received in the storage folder, the new radar echo data is segmented, and multiple computing units are obtained after the segmentation; specifically including:

[0069] Step S210: If the new radar echo data is received in the storage folder, determine whether the data coverage area of ​​the new radar echo data is an integer multiple of the preset cutting area;

[0070] Step S220: If it is determined that the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is filled.

[0071] Furthermore, in this step, in order to facilitate model calculation and cutting, the size of the original data coverage area should be an integer multiple of the cutting area. Therefore, before cutting, it should be determined whether the size of the original data coverage area is an integer multiple of the cutting area. If not, it should be filled.

[0072] Step S230: Perform data segmentation on the newly filled radar echo data, and obtain multiple computing units after the segmentation is completed.

[0073] In one embodiment, step S220: If it is determined that the size of the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is being filled. The height and width of each area, the number of cutting areas, and the relationship after expansion are as follows:

[0074]

[0075] Where cwn is the number of horizontal segments, chn is the number of vertical segments, roundup is the round-up operation, ow is the width of the original data coverage area, cw is the width of the prediction area, oh is the height of the original data coverage area, ch is the height of the prediction area, h is the height after expansion, and w is the width after expansion.

[0076] For example, the coverage area of ​​the weather radar echo data used in this invention is from 73.00° to 135.00° east longitude and from 12.20° to 54.20° north latitude.

[0077] The latitude and longitude coordinates are accurate to 0.01° and arranged in a 4200x6200 matrix. The data input to the model is 512x512. Since 4200 and 6200 are not integer multiples of 512, the data is first padded with zeros. According to the formula above, the number of rows is expanded from 4200 to 4608, and the number of columns is expanded from 6200 to 6656. The filling effect is as follows. Figure 3 As shown.

[0078] According to the formula above, the original data coverage area can be divided into cwn*chw data units.

[0079] For example, the weather radar echo data used in this invention can be divided into 9x13 (9 columns and 13 rows) after being filled, for a total of 117 calculation units.

[0080] 1 10 19 28 37 46 55 64 73 82 91 100 109 2 11 20 29 38 47 56 65 74 83 92 101 110 3 12 21 30 39 48 57 66 75 84 93 102 111 4 13 22 31 40 49 58 67 76 85 94 103 112 5 14 23 32 41 50 59 68 77 86 95 104 113 6 15 24 33 42 51 60 69 78 87 96 105 114 7 16 25 34 43 52 61 70 79 88 97 106 115 8 17 26 35 44 53 62 71 80 89 98 107 116 9 18 27 36 45 54 63 72 81 90 99 108 117

[0081] When dividing the data, the matrix row and column formulas for calculating the cells can be obtained by using the cell numbers, as follows:

[0082]

[0083] Where rowstart is the starting position of the row of the matrix, rowend is the ending position of the row of the matrix, columnstart is the starting position of the column of the matrix, columnend is the ending position of the column of the matrix, index is the number of the computation unit, and hlen is the number of blocks in the vertical direction.

[0084] In the segmentation method of this invention, hlen is 9, width is the number of rows covered by the calculation unit, height is the number of columns covered by the calculation unit, both are 512, % is the modulo operation, and / is the integer operation.

[0085] For example, the calculation unit numbered 25 can be divided into regions using the formula above. The row range is from 3072 to 3584, and the column range is from 1024 to 1536.

[0086] Furthermore, the size of the partition matrix is The four-dimensional matrix, where cn is the number of computational units and inputnum is the length of the input data for each computational unit. It is the size of the computing unit.

[0087] In one embodiment, step S300 involves storing the segmented computing units and dynamically allocating the stored computing units to each precipitation prediction processing machine. After allocation, each precipitation prediction processing machine receives the same amount of data from its allocated computing units. Specifically, this includes:

[0088] Step S310: Store the segmented data of each computing unit into a preset segmentation matrix;

[0089] Step S320: Obtain the amount of data contained in each of the stored computing units;

[0090] Step S330: Obtain the pre-set number of precipitation prediction processing machines;

[0091] Step S340: Allocate each computing unit to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, wherein, after the allocation is completed, the amount of data allocated to each computing unit by each precipitation prediction processing machine is the same.

[0092] Furthermore, in this embodiment, since each computing unit corresponds to a different geographical region, and the region with more precipitation has a large amount of data and a long computing time, dynamic allocation is performed in order to distribute the data units evenly across the machine for prediction. Dynamic allocation means that the data units are evenly distributed across the machine for prediction based on the amount of data on the machine and the amount of data in each computing unit.

[0093] Since data with no precipitation is represented as 0, the amount of data in each computational unit can be represented by the proportion of non-zero data. The proportion of non-zero data (dr) in each computational unit is calculated as follows:

[0094]

[0095] in, It counts the number of non-zero data points. It is the total number of each computing unit.

[0096] In one embodiment, step S340: allocating each computing unit to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, specifically includes:

[0097] Step S341: Randomly select n initial computing units from the set data of computing units and assign them to n precipitation prediction processing machines, wherein the set data of computing units contains all computing units;

[0098] Step S342: Sort the n precipitation prediction and processing machines in ascending order of the amount of data they possess;

[0099] Step S343: Randomly select n computing units from the set of computing units data and assign them to n precipitation prediction processing machines;

[0100] Step S344: Repeat steps S341-S343 until the amount of data allocated to each of the precipitation prediction processing machines is the same, at which point the allocation ends.

[0101] Furthermore, in this embodiment, the data proportion of each computing unit is different. In order to make the amount of data allocated to each computing unit on each machine as even as possible, the above steps S341-S344 are used.

[0102] Through the above allocation steps, each precipitation prediction processing machine obtains the number of the computing unit it is responsible for. Each precipitation prediction processing machine obtains the corresponding input data from the segmentation matrix according to the computing unit number it is responsible for, and begins to make predictions.

[0103] In one embodiment, step S400: Based on a preset spatiotemporal prediction model, precipitation prediction is performed on each of the computing units, and precipitation prediction results are generated for each computing unit. The precipitation prediction results are then merged to generate precipitation prediction data, specifically including:

[0104] Step S410: After the calculation unit is allocated to each precipitation prediction processing machine, a prediction start command is sent to each precipitation prediction processing machine;

[0105] Step S420: According to the prediction start command, control each precipitation prediction processing machine to perform precipitation prediction on the calculation unit based on the spatiotemporal prediction model, and generate precipitation prediction results for each calculation unit respectively;

[0106] Step S430: Check whether all the precipitation prediction processing machines have completed their predictions;

[0107] Step S440: If all the precipitation prediction processing machines have completed their predictions, then the precipitation prediction results are merged and precipitation prediction data is generated.

[0108] Furthermore, in this step, the spatiotemporal prediction model of the present invention is a four-layer spatiotemporal prediction neural network composed of spatiotemporal memory units, the structure of which is as follows: Figure 4 As shown, Figure 4 The horizontal axis of the network is the accumulation of the time memory C of the previous n time steps, and the vertical axis is the accumulation of the memory M of different spaces at the current time step.

[0109] To ensure that spatial memories at different times are sequential, the last layer of spatial memory from the previous moment is used as the first layer of spatial memory for the next moment. H serves as hidden information containing both temporal memory C and spatial memory.

[0110] Furthermore, the spatiotemporal prediction model of the present invention uses 1 hour of historical radar echo data as input and 2 hours of radar echo data as output.

[0111] Each machine invokes the model to make predictions for the corresponding computing unit.

[0112] The size of the prediction matrix is A four-dimensional matrix, where cn is the number of computational units. is the size of the computation unit, and outputnum is the length of the output data of each computation unit.

[0113] The prediction results of this invention span a time period of 2 hours, resulting in a total of 20 radar echo data points. The processing time of each precipitation prediction processing machine is inconsistent. If all predictions are not completed before a new data point arrives, it will lead to errors in the prediction results. Furthermore, before proceeding to the next merging operation, it is essential to ensure that the prediction tasks of all computing units are completed.

[0114] Therefore, the scheduling system has a machine state array, the length of which is the same as the number of machines scheduled by the distributed scheduling system. This invention relates to 8 precipitation prediction processing machines, so the length of the state array is 8. 0 represents incomplete, and 1 represents completed. When all states are 1, it means all machines have completed the prediction task; if not all states are 1, it means the current prediction round is still in progress.

[0115] When all values ​​are 1, the scheduling system is notified, and the system then sets all values ​​to 0. The matrix formula for calculating the unit number is used, and the corresponding result for each number is placed in the calculated position and merged, thus completing the entire scheduling process.

[0116] In one embodiment, the segmentation module is further configured to:

[0117] The segmented data of each computing unit is stored in a preset segmentation matrix; the amount of data contained in each of the stored computing units is obtained; the number of precipitation prediction processing machines is obtained in advance; each computing unit is allocated to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, wherein, after the allocation is completed, the amount of computing unit data allocated to each precipitation prediction processing machine is the same.

[0118] The merging module is further configured to: after the computing units are allocated to each precipitation prediction processing machine, send a prediction start command to each precipitation prediction processing machine; control each precipitation prediction processing machine to perform precipitation prediction on the computing units based on the spatiotemporal prediction model according to the prediction start command, and generate precipitation prediction results for each computing unit respectively; detect whether all precipitation prediction processing machines have completed the prediction; if all precipitation prediction processing machines have completed the prediction, merge the precipitation prediction results and generate precipitation prediction data.

[0119] In one embodiment, the segmentation module is further configured to perform the following steps:

[0120] Step 1: Randomly select n initial computing units from the set of computing units data and assign them to n precipitation prediction processing machines, wherein the set of computing units data contains all computing units;

[0121] Step 2: Sort the n precipitation prediction and processing machines in ascending order of the amount of data they possess;

[0122] Step 3: Randomly select n computing units from the set of computing units data and assign them to n precipitation prediction and processing machines;

[0123] Step S344: Repeat steps one through three until the amount of data allocated to each of the precipitation prediction processing machines is the same, at which point the allocation ends.

[0124] In one embodiment, the segmentation module is further configured to: if the new radar echo data is received in the storage folder, determine whether the data coverage area size of the new radar echo data is an integer multiple of a preset segmentation area; if the data coverage area size of the new radar echo data is not an integer multiple of a preset segmentation area, fill the new radar echo data; segment the filled new radar echo data, and obtain multiple computing units after the segmentation is completed.

[0125] If the size of the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is being filled, and the relationship after filling is as follows:

[0126]

[0127] Where cwn is the number of horizontal segments, chn is the number of vertical segments, roundup is the round-up operation, ow is the width of the original data coverage area, cw is the width of the prediction area, oh is the height of the original data coverage area, ch is the height of the prediction area, h is the height after expansion, and w is the width after expansion.

[0128] In one embodiment, such as Figure 5 As shown, a computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps described above in the dynamic distributed short-term scheduling method for short-term precipitation forecasting.

[0129] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described above in the dynamic distributed short-term scheduling method for short-term precipitation forecasting.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A dynamic distributed short-term scheduling method for short-term precipitation forecasting, characterized in that, The method includes: Step S100: Check if new radar echo data has been received in the preset storage folder; Step S200: If the new radar echo data is received in the storage folder, the new radar echo data is segmented, and multiple computing units are obtained after the segmentation. The steps include: Step S210: If the new radar echo data is received in the storage folder, determine whether the data coverage area of ​​the new radar echo data is an integer multiple of the preset cutting area; Step S220: If it is determined that the data coverage area of ​​the new radar echo data is not an integer multiple of the preset cutting area, then the new radar echo data is padded, and the relationship after padding is as follows: ； ； ； ； Where cwn is the number of horizontal segments, chn is the number of vertical segments, roundup is the round-up operation, ow is the width of the original data coverage area, cw is the width of the prediction area, oh is the height of the original data coverage area, ch is the height of the prediction area, h is the height after expansion, and w is the width after expansion. Step S230: Perform data segmentation on the newly filled radar echo data, and obtain multiple computing units after segmentation; Step S300: Store the segmented computing units and dynamically allocate the stored computing units to each precipitation prediction processing machine. After allocation, each precipitation prediction processing machine is allocated the same amount of computing unit data. The steps include: Step S310: Store the segmented data of each computing unit into a preset segmentation matrix; Step S320: Obtain the amount of data contained in each of the stored computing units; Step S330: Obtain the pre-set number of precipitation prediction processing machines; Step S341: Randomly select n initial computing units from the set data of computing units and assign them to n precipitation prediction processing machines, wherein the set data of computing units contains all computing units; Step S342: Sort the n precipitation prediction and processing machines in ascending order of the amount of data they possess; Step S343: Randomly select n computing units from the set of computing units data and assign them to n precipitation prediction processing machines; Step S344: Repeat steps S341-S343 until the amount of data allocated to each of the precipitation prediction processing machines is the same, at which point the allocation ends; Step S400: Based on the preset spatiotemporal prediction model, precipitation prediction is performed on the calculation units respectively, and precipitation prediction results of each calculation unit are generated. The precipitation prediction results are then merged to generate precipitation prediction data.

2. The dynamic distributed short-term scheduling method for short-term precipitation forecasting according to claim 1, characterized in that, Step S400: Based on a preset spatiotemporal prediction model, precipitation prediction is performed on each of the computing units, and precipitation prediction results are generated for each computing unit. The precipitation prediction results are then merged to generate precipitation prediction data, specifically including: Step S410: After the calculation unit is allocated to each precipitation prediction processing machine, a prediction start command is sent to each precipitation prediction processing machine; Step S420: According to the prediction start command, control each precipitation prediction processing machine to perform precipitation prediction on the calculation unit based on the spatiotemporal prediction model, and generate precipitation prediction results for each calculation unit respectively; Step S430: Check whether all the precipitation prediction processing machines have completed their predictions; Step S440: If all the precipitation prediction processing machines have completed their predictions, then the precipitation prediction results are merged and precipitation prediction data is generated.

3. A dynamic distributed short-term scheduling system for short-term precipitation forecasting, used to execute the method as described in any one of claims 1 to 2, characterized in that, The system includes: The file receiving module is used to query whether new radar echo data has been received in a preset storage folder; A segmentation module is used to: segment the new radar echo data received in the storage folder, and obtain multiple computing units after segmentation; store the segmented computing unit data into a preset segmentation matrix; obtain the amount of data contained in each of the stored computing units; obtain the number of pre-set precipitation prediction processing machines; allocate each computing unit to each precipitation prediction processing machine according to the amount of data contained in each computing unit and the number of precipitation prediction processing machines, wherein after allocation, each precipitation prediction processing machine is allocated the same amount of data in each computing unit; after the computing units are allocated to each precipitation prediction processing machine, send a prediction start command to each precipitation prediction processing machine; control each precipitation prediction processing machine to perform precipitation prediction on the computing units based on the spatiotemporal prediction model according to the prediction start command, and generate precipitation prediction results for each computing unit respectively; detect whether all precipitation prediction processing machines have completed prediction; if all precipitation prediction processing machines have completed prediction, merge the precipitation prediction results and generate precipitation prediction data. The merging module is used to perform precipitation prediction on the calculation units based on a preset spatiotemporal prediction model, generate precipitation prediction results for each calculation unit, and merge the precipitation prediction results to generate precipitation prediction data.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 2.

Images