A method and system for identifying extreme weather at the edge using microwave radiometer data

Abstract

The present application discloses an edge-side extreme weather identification method applicable to microwave radiometer data, which belongs to the field of remote sensing technology. It addresses the problem of low bandwidth and lack of real-time performance of satellite-to-ground communication in the existing technology. The method of the present application obtains brightness temperature data from a remote sensing satellite equipped with a microwave radiometer, filters out the corresponding remote sensing data packet, and constructs a microwave image containing multiple microwave bands and polarization information through data unpacking, brightness temperature data processing and image generation steps. According to the type and number of extreme weather that needs to be identified, a suitable network architecture is built for model training to generate an extreme weather identification model. Subsequently, quantization technology is used to optimize the trained model to ensure efficient operation on edge devices. The advantages of the present application are that it is efficient and practical, and is suitable for executing complex deep learning tasks in resource-constrained edge computing devices, providing an effective technical means for real-time monitoring of extreme weather.

Description

Technical Field

[0001] The present invention relates to an edge-end extreme weather recognition method and system applicable to microwave radiometer data, belonging to the field of remote sensing technology. Background Art

[0002] The microwave radiometer is a passive microwave remote sensing payload with a wide swath and an observation band that can penetrate clouds. It has all-weather observation capabilities, has great advantages in identifying extreme weather, and can play a significant role in disaster prevention and mitigation.

[0003] Currently, a large number of extreme weather identifications use deep learning technology based on visible light imaging. However, the information contained in passive microwave imaging is the brightness temperature emitted by the object, while the information contained in visible light imaging is the intensity of the object's reflection of visible light in different bands. This difference leads to differences in the data preprocessing and training process of the deep learning model.

[0004] Deep learning techniques based on visible light imaging are widely used in the field of extreme weather identification. However, this method has significant limitations: its reliance on sunlight reflection prevents it from operating at night (e.g., it completely fails in the case of a typhoon making landfall at night); visible light cannot penetrate clouds, resulting in significant cloud interference when observing weather phenomena unrelated to clouds; it also lacks three-dimensional atmospheric thermal information, making it unable to effectively capture the temperature and humidity structure of the core area of ​​heavy rain convection, and prone to high misjudgment rates in snowy and icy areas. In contrast, passive microwave radiometers, by receiving the thermal radiation of ground objects themselves, offer unique advantages in cloud penetration, all-day operation, and three-dimensional atmospheric detection, providing a physical foundation for quantitative identification of extreme weather. Despite this, applying microwave data to deep learning still faces significant challenges: its low spatial resolution and multi-band raster characteristics significantly differ from high-resolution RGB three-channel visible light images; microwave brightness temperature values ​​are affected by factors such as surface emissivity and atmospheric attenuation, resulting in complex physical meanings, making it difficult for models to effectively extract key weak signal features; and, critically, the scarcity of extreme weather samples exacerbates the risk of model overfitting. Existing deep learning methods based on microwave data face significant bottlenecks: the original FP32 model of the neural network's massive computational load cannot meet the real-time requirements of onboard processors, limiting its deployment capabilities on the front lines of disaster prevention and mitigation; and post-training quantization (PTQ) calibration using random samples can lead to higher underreporting rates compared to the FP32 model in cases of severe class imbalance. This application addresses these pain points by innovatively proposing an edge-based extreme weather recognition method for microwave radiometer data. Summary of the Invention

[0005] In response to the problems existing in the existing technology, this application proposes an edge extreme weather identification method and system suitable for microwave radiometer data. It is efficient and practical, suitable for performing complex deep learning tasks in resource-constrained edge computing devices, and provides an effective technical means for real-time monitoring of extreme weather.

[0006] To solve the above technical problems, the present invention adopts a technical solution that is an edge extreme weather identification method applicable to microwave radiometer data, comprising the following steps:

[0007] 1) Obtain microwave radiometer remote sensing data;

[0008] 2) Filtering the microwave radiometer remote sensing data obtained in step 1) and creating a basic data set based on the time and longitude and latitude of recorded extreme weather events;

[0009] 3) Build a convolutional neural network based on the number and types of extreme weather events to be identified; train the convolutional neural network and derive an extreme weather recognition model;

[0010] 4) Screening the data set used for the quantitative model and quantifying the exported model;

[0011] 5) Use extreme weather recognition models to identify extreme weather.

[0012] Optimized, the above-mentioned edge extreme weather identification method applicable to microwave radiometer data, in step 1), remote sensing data is obtained from a remote sensing satellite equipped with a microwave radiometer, and the remote sensing data includes brightness temperature information, timestamp, and latitude and longitude;

[0013] In step 2), according to the recorded time and longitude and latitude of the extreme weather, in the process of screening the microwave radiometer remote sensing data obtained in step 1), positive sample data and negative sample data are screened;

[0014] The specific process of screening positive sample data includes:

[0015] 201) Filtering remote sensing data packets containing brightness temperature information according to the time and longitude and latitude of extreme weather occurrence, and unpacking the data packets;

[0016] 202) Filtering scanning points located in extreme weather areas based on latitude and longitude information, assigning these scanning points to multiple layers according to microwave bands and polarization modes, and generating microwave images containing multiple band and polarization information;

[0017] 203) Convert the brightness temperature data of the scan points to Kelvin and store them in the HDF5 file in float16 format;

[0018] The process of screening negative sample data includes:

[0019] 204) selecting a time and area where the extreme weather to be identified does not exist, and repeating steps 201) to 204) to generate negative sample data;

[0020] The process of making a basic dataset includes:

[0021] 205) The microwave images with and without extreme weather are stored in folders with different names, and the folder names are used as data labels.

[0022] Optimized, the above-mentioned edge extreme weather identification method applicable to microwave radiometer data, after step 2) is completed, the basic data set is enhanced, including scaling, rotating, stretching, and cropping or padding the microwave image in the first and second dimensions, and the expanded data set is stored in the storage format of the basic data set.

[0023] The above-mentioned edge extreme weather recognition method applicable to microwave radiometer data is optimized. In step 4), the various types of extreme weather and common non-extreme weather images in the dataset account for an equal proportion, and the total amount of data is 10% of the total training set; the floating-point model is quantized to an int8 model, and the quantized dataset is imported during the quantization process, the activation value and weight information are counted, the scale factor and zero point are determined, and then verification and fine-tuning are performed.

[0024] Optimized, the above-mentioned edge extreme weather recognition method applicable to microwave radiometer data, in step 201), let the time when extreme weather occurs be t, and the longitude and latitude of the extreme weather center at that time be [λ, φ], then assuming that at [t min ,t max ] time range, the range of extreme weather is {[λ min ,λ max ]、[φ min ,φ max ]}, where t min =t-15min, t max =t+15min;λ min =λ-7.5°,λ max =λ+7.5°;φ min =φ-7.5°,φ max =φ+7.5°; the meanings of these variables are: λ: longitude of the extreme weather center; φ: latitude of the extreme weather center; t min : the earliest time when extreme weather occurs; t max : the moment when extreme weather disappears; λ min : the westernmost longitude of the extreme weather area; λ max : the easternmost longitude of the extreme weather area; φ min: the southernmost latitude of the extreme weather area; φ max : The northernmost latitude of the area where extreme weather occurs.

[0025] For the downloaded remote sensing data package, first analyze the file name and select the acquisition time t i Satisfy t min <t i <t max Data packets, record the names of these data packets in an array, and open the data packets in sequence;

[0026] In step 202), based on the time and longitude and latitude of the extreme weather, remote sensing data packets containing extreme weather are screened out, the data packets are unpacked, and brightness temperature information is extracted;

[0027] Brightness temperature information includes the brightness temperature of the five channels of the microwave radiometer: 18.7 GHz-H channel brightness temperature, 18.7 GHz-V channel brightness temperature, 23.8 GHz-V channel brightness temperature, 37 GHz-H channel brightness temperature, and 37 GHz-V channel brightness temperature. The brightness temperature data of these channels are resampled to the same set of grid points to ensure that there are brightness temperature data of five channels at each longitude and latitude.

[0028] According to the longitude and latitude of the extreme weather eye, all scanning points within a square longitude and latitude grid with a span of 15° centered on that longitude and latitude are selected;

[0029] In step 203), the brightness temperature data of the selected scanning points are converted from count values ​​to Kelvin; the extreme weather area, i.e. {longitude∈[λ min ,λ max ], latitude∈[φ min ,φ max ]}, all scanning points are arranged according to the longitude and latitude information to obtain a set of three-dimensional arrays; the obtained three-dimensional arrays are stored in HDF5 files in float16 format; the above operations are performed on the time and longitude and latitude areas where multiple extreme weather conditions exist to obtain multiple HDF5 files, which constitute the positive sample data.

[0030] In step 204), the microwave radiometer raw data is filtered according to the time and area that do not contain extreme weather, and steps 201) to 204) are repeated to generate negative sample data.

[0031] Optimized, the above-mentioned edge extreme weather recognition method for microwave radiometer data, after step 2) is completed, when performing data augmentation on the basic data set, the images in the basic data are randomly scaled within the range of [-0.2, 0.2]; the images are randomly rotated within the range of [-90°, 90°]; the images are randomly stretched in the X direction or the Y direction; and finally, the images are padded with zeros to make the final image rectangular;

[0032] The enhanced data storage format is consistent with the basic dataset.

[0033] Optimized, the above-mentioned edge extreme weather identification method applicable to microwave radiometer data, in step 3), according to the number and type of extreme weather to be identified, when building a convolutional neural network, a deep learning model is constructed based on the residual network;

[0034] The residual network is built based on basic-1block and basic-2block;

[0035] The input matrix is ​​scaled to 224*224 pixels in the first two dimensions, and then passes through a 7*7 conv kernel, a Maxpool layer, a basic-1 block, three basic-2 blocks, and finally an FC layer;

[0036] In step 3), the convolutional neural network is trained using the base and augmented datasets. The training process includes forward propagation, backpropagation, and optimization, using the Cross-Entropy loss function and the Adam optimizer. The training process is organized into multiple epochs, each consisting of multiple iterations.

[0037] Optimized, the above-mentioned edge extreme weather recognition method applicable to microwave radiometer data, in step 4), when the exported model is quantized, when a piece of data is input into the neural network built in step 3), if the output value of a neuron in the last layer (FC layer) of the network is greater than 0.85, then the data is representative data; all representative data in the training set are screened as candidate data sets for quantization;

[0038] Assume there are k labels, among which N labels are selected from the i-th label. i representative data, then N min =min(N1,N2,…,N i ,…,N k ), randomly select N from the representative data of each label mindata as a quantized dataset; the original model is converted from float32 to int8 format using tensorflow's PTQ (Post-Training Quantization) quantization tool. During the quantization process, the vectorization tool inputs the quantized dataset, and the tool automatically counts the dynamic range distribution of activation values ​​and weights of each layer. Based on the statistical results, the scale factor and zero point of each layer are calculated based on the asymmetric quantization scheme. Then, the original FP32 model weights and activation values ​​are quantized according to the following formula: Q = round (FP32 / S) + Z, where S is the scale factor and Z is the zero point.

[0039] An edge extreme weather recognition system for microwave radiometer data. When building the system, a deep learning processor suitable for the aforementioned edge extreme weather recognition method is selected, and a compiler suitable for the processor is used to compile the model.

[0040] The compiled model is imported into the deep learning dedicated processor on the edge; the model is run to perform extreme weather identification on the edge device.

[0041] The above-mentioned edge extreme weather recognition system for microwave radiometer data is optimized.

[0042] When selecting a deep learning processor, configure a DPU (Deep-learning Processor Unit) adapted to the Zynq chip; add DPUCZDX8G to the Vivado project and select the B4096 architecture; adjust the DPU's AXI interface bandwidth based on the Zynq PS architecture; and enable support for the leaky-relu layer in the DPU configuration file.

[0043] After the synthesis step, in the floorplanning tool, select the location on the FPGA chip closest to the PS end for the DPU. In the implementation strategy options, select the Explore strategy and attach the -SSI_HighUtilSLRs directive.

[0044] After completing the IP configuration, generate a bitstream file (.bit) containing the DPU IP core;

[0045] When using the petalinux configuration kernel to create a Linux system, configure the kernel, add driver support for the DPU, and add the DPU path in the device tree;

[0046] After the compilation is complete, the system image BOOT.BIN and image.ub containing the DPU driver are generated, and the system image is burned to the EMMC (Embedded MultiMediaCard) memory of the ZYNQ chip;

[0047] In the Linux system running on the Zynq chip, the Vitis-AIRuntime API is called to load the model (.xmodel), and the input image is scaled, normalized, and converted to the DPU format. A kernel-mode continuous memory buffer is allocated, and the DPU is driven by the VART API to perform graph operations, realizing the input→inference→output data flow.

[0048] The dual-core CPU performs asynchronous pipeline control to build a three-stage pipeline of preprocessing → transmission → inference: CPU core 1 manages DPU communication and memory buffering, and CPU core 2 performs data preprocessing.

[0049] The beneficial effects of this application are:

[0050] To address the shortcomings of existing technologies, this application proposes a method and system for edge-based extreme weather identification using microwave radiometer data. The advantages of this method include: ① Using microwave radiometer data to identify extreme weather addresses the issue of visible light observation equipment being unable to observe on the side of the Earth facing away from the sun and being susceptible to cloud interference. ② Building a convolutional network and using microwave radiometer observation data from multiple frequency bands as network input improves the accuracy of recognition results. ③ Based on the time and longitude and latitude records of extreme weather occurrences, a basic dataset of positive and negative samples is automatically generated. An appropriate data format is selected based on the characteristics of microwave radiometer data, improving dataset generation and storage efficiency. This lays the foundation for improving network accuracy. ④ During training, random rotation, scaling, and stretching of multidimensional matrices are utilized to enhance network robustness. ⑤ Based on the characteristics of PTQ quantization, a method for generating quantized datasets is proposed. This allows the quantized int8 format neural network model to significantly reduce computational complexity while maintaining similar accuracy to the original float32 format neural network, facilitating deployment on edge devices. ⑥ When laying out the FPGA chip, place the DPU (Deep-learning Processing Unit) in the area closest to the PS (Processing System) to increase the probability of successful wiring and provide computing device support for extreme weather identification at the edge. ⑦ In the process of processing the neural network by the edge chip (ZYNQ), the first CPU core of the PS is responsible for data interaction and memory management with the DPU, and the second CPU core is responsible for data preprocessing. While improving data throughput, the operating system will not crash due to the large amount of data preprocessing calculations resulting in the key process not being able to allocate CPU computing resources, thereby improving the stability of the edge computing system. ⑧ Through the above technology, the present invention trains a deep learning network capable of identifying extreme weather and deploys the network to the edge system, reducing the communication frequency and amount of communication data between the edge system and users, and improving the real-time performance of extreme weather identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of the process of an edge extreme weather identification method provided in an embodiment;

[0052] Figure 2 This is the block component of the edge extreme weather recognition model provided in the embodiment - basic-1block;

[0053] Figure 3 This is the block component of the edge extreme weather recognition model provided in the embodiment - basic-2block;

[0054] Figure 4This is an overall structural diagram of the edge extreme weather recognition model provided in the embodiment. DETAILED DESCRIPTION

[0055] The technical features of the present invention are further described below with reference to specific embodiments.

[0056] One of the technical solutions of the edge extreme weather identification method applicable to microwave radiometer data in this application uses the DPU running on the ZYNQ system as a dedicated deep learning processor and uses microwave radiometer data for edge typhoon identification. The specific steps are as follows.

[0057] First, we need to preprocess the microwave radiometer data. Remote sensing data is obtained from remote sensing satellites equipped with microwave radiometers. This data includes brightness temperature information, timestamps, and longitude and latitude.

[0058] Screening positive sample data, the specific process includes

[0059] Filter remote sensing data packages containing typhoon weather.

[0060] Assume that the time when the typhoon appears is t, and the longitude and latitude of the typhoon center at that time is [λ, φ]. min ,t max ] time range, the typhoon exists within the range of {[λ min ,λ max ],[φ min ,φ max ]}, where t min =t-15min, t max =t+15min;λ min =λ-7.5°,λ max =λ+7.5°;φ min =φ-7.5°,φ max =φ+7.5°. For the downloaded remote sensing data package, first analyze the file name and select the acquisition time t i Satisfy t min <t i <t max The data packets are recorded in an array and opened in sequence.

[0061] Based on the time and longitude of the typhoon's appearance, remote sensing data packets containing typhoon weather data were selected, unpacked, and brightness temperature information extracted. This brightness temperature information requires the brightness temperature of the five channels of the microwave radiometer: 18.7 GHz-H channel brightness temperature, 18.7 GHz-V channel brightness temperature, 23.8 GHz-V channel brightness temperature, 37 GHz-H channel brightness temperature, and 37 GHz-V channel brightness temperature. These channel brightness temperature data must be resampled to the same set of grid points to ensure that all five channels have brightness temperature data at each longitude and latitude. Based on the longitude and latitude of the typhoon eye, all scanning points within a square longitude and latitude grid with a span of 15°, centered at that longitude and latitude, were selected.

[0062] Convert the brightness temperature data of the selected scan points from count values ​​to Kelvin to unify the data units. Arrange all scan points within the range according to their latitude and longitude information to obtain a three-dimensional array. Store this array in an HDF5 file in float16 format to efficiently store and organize large datasets. Repeat this operation for multiple typhoon-related time periods and latitude and longitude regions to generate multiple HDF5 files, which constitute the positive sample data.

[0063] Filter negative sample data.

[0064] Filter the raw microwave radiometer data based on time and region that do not contain typhoons, and repeat the above steps to generate negative sample data. Store this data in a folder different from the positive sample data. The folder name will be the data label.

[0065] Data augmentation.

[0066] Data augmentation is performed on the base dataset to increase data diversity and model generalization. The following operations are performed on the base dataset: random scaling within the range [-0.2, 0.2]; random rotation within the range [-90°, 90°]; random stretching in the X or Y direction; and finally, zero padding to make the final image rectangular. The augmented data is stored in the same format as the base dataset.

[0067] Build deep learning models.

[0068] The deep learning model is based on a residual network, which is constructed using basic-1block and basic-2block. The input matrix is ​​resized to 224x224 pixels in the first two dimensions and then passes through a 7x7 convolution kernel, a maxpool layer, one basic-1block, three basic-2blocks, and finally an FC layer.

[0069] The basic-1 block is composed of two identical subcomponents. The first layer is a 3*3 convolutional layer with a stride of 1 and a patch size of 1. It is followed by a BatchNormalization layer to accelerate convergence. It is then followed by a Leaky-Relu layer with α set to 0.1 to prevent neuron "death". The expression of the Leaky-Relu layer is:

[0070]

[0071] Here, α is a small positive number. The second layer is a 3x3 convolutional layer with a stride of 1 and a patch size of 1. It is followed by a BatchNormalization layer to accelerate convergence. The output of this BatchNormalization layer has the same shape as the input of this submodule. Their corresponding positions are added together to form a subcomponent of the basic-1 block. The two subcomponents are cascaded to form the output of the basic-1 block.

[0072] The basic-2block consists of two different subcomponents. The first subcomponent consists of a 3x3 convolutional layer with a stride of 2, followed by a batch normalization layer and a leaky-relu layer. The second subcomponent consists of a 3x3 convolutional layer followed by a batch normalization layer. At the input of the module, a 1x1 convolutional layer with a stride of 1 is connected, followed by a batch normalization layer. The output is then added to the output of the second layer and then passed through a leaky-relu layer. The third subcomponent consists of a 3x3 convolutional layer with a stride of 1, followed by a batch normalization layer and a leaky-relu layer. The fourth subcomponent consists of a 3x3 convolutional layer with a stride of 1, followed by a batch normalization layer. The output of this layer is added to the output of the leaky-relu layer following the second layer and then passed through a leaky-relu layer to produce the output of the basic-2block.

[0073] Deep learning model training.

[0074] The model is trained using the base and augmented datasets. The training process includes forward propagation, backpropagation, and optimization, using the Cross-Entropy loss function and the Adam optimizer. The training process is organized into multiple epochs, each of which contains multiple iterations. In each iteration, a shuffled mini-batch of data is extracted from the dataset, and forward propagation, backpropagation, and parameter updates are performed. In addition, the performance of the model on the validation set is evaluated at the end of each epoch to monitor the convergence of the model. When the loss function does not improve significantly within 10 epochs, the learning rate is decreased. If the loss function still does not change significantly after 10 epochs after decreasing the learning rate, training is stopped to prevent overfitting. Finally, the model weights are exported to a file in a TensorFlow-specific format.

[0075] Finally, the model is quantified and deployed, and the specific process includes

[0076] Representative data was selected from the training set as the quantization dataset, with an equal number of images selected from the folders representing each label. The original model was converted from float32 to int8 format using the quantization tool. During the quantization process, the quantized dataset was input, activation values ​​and weight information were counted, and the quantization scale factor and zero point were determined. Quantization verification and fine-tuning were then performed to ensure that model accuracy was not significantly affected. Finally, to run the quantized model on the Xilinx ZYNQ DPU, the quantized model was compiled using the compiler provided by Vitis-ai with a finger-point consistent with the DPU. The compiled model file (.XMODEL) was stored in the ZYNQ system, using the DPU to accelerate model inference calculations.

[0077] Model quantization and deployment.

[0078] Use Vitis-AI for model quantization and fine-tuning, configure the DPU IP core on the Xilinx ZYNQ chip, support the DPU driver through PetaLinux, transmit model and test data, and use VART in the ZYNQ Linux system to accelerate the inference of convolutional neural networks and store the inference results.

[0079] Model quantization.

[0080] Model quantization converts the original high-precision floating-point model to a low-precision integer model to reduce model size and accelerate inference speed. Representative data is selected from the training set as the quantization dataset, ensuring that an equal number of images are selected from each category folder to ensure balanced features across categories during the quantization process. A configuration file containing quantization parameters is created, including quantization accuracy, calibration method, quantization dataset path, and model path. Using the vai_q_tensorflow tool in Vitis-AI, run the quantization command. The PTQ (Post-Training Quantization) tool converts the weights and activation values ​​in the original float32 model to int8 format. During this process, the tool uses the quantized dataset to calculate activation and weight information and determine the scaling factor and zero point for each layer to maintain the accuracy stability of the quantized model.

[0081] Model fine-tuning.

[0082] After quantization, the model's accuracy may decline, so fine-tuning is necessary to restore or improve performance. First, set the hyperparameters required for fine-tuning, including the learning rate, number of training rounds, and optimizer type. Use the vai_finetune tool to execute the fine-tuning instructions. During fine-tuning, the model continues to train on the quantized dataset to adjust the weights to minimize the loss of accuracy during inference for the quantized int8 model. After fine-tuning, evaluate the fine-tuned model on the validation set. If the model does not meet the expected accuracy or deviates significantly from the original float model's accuracy, continue iterating the fine-tuning process.

[0083] Configure the DPU IP core adapted for ZYNQ.

[0084] Add DPUCZDX8G to the Vivado project and select the B4096 architecture; adjust the AXI interface bandwidth of the DPU according to the PS-side architecture of ZYNQ; in addition, it is necessary to enable support for the leaky-relu layer in the DPU configuration file. After the synthesis step, in the floorplanning, select the location closest to the PS end on the FPGA chip for the DPU to increase the probability of successful routing and the DPU frequency. In the implementation strategy options, select the `Explore` strategy to perform multiple optimizations; it is also necessary to add the `-SSI_HighUtilSLRs` directive to make the layout more compact and reserve enough space for other logic content. After completing the IP configuration, generate a bitstream file (.bit) containing the DPU IP core for subsequent integration in PetaLinux.

[0085] When creating a Linux system using the petalinux kernel, you need to configure the kernel, add driver support for the DPU, and add the DPU path to the device tree. After compilation, a system image (BOOT.BIN and image.ub) containing the DPU driver is generated for flashing to the Zynq chip's EMMC memory.

[0086] Transfer of model and test data.

[0087] Transfer the quantized model file (.xmodel) and test data to the Linux system of the ZYNQ chip via Ethernet. Place the ZYNQ and host on the same network segment of the same router. Use FTP on the host to transfer the .xmodel file and test data to the target directory on the ZYNQ.

[0088] Use VART for DPU acceleration in ZYNQ's Linux system.

[0089] In the Zynq Linux system, Vitis-AIRuntime (VART) is used to control the transmission of deep learning data between the PS and PL, and the DPU is used to accelerate inference of convolutional neural networks. The Zynq Linux system is rooted, and the VART API is used in the application to load the transferred .xmodel file. The input image is preprocessed according to the model requirements, including scaling and normalization, and converted into a format suitable for DPU input. Memory buffers (kernel state) are allocated for input and output to ensure that the memory space used by the PL can be continuously accessed by the PL's DMA, improving access speed. The preprocessed input data is transferred to the DPU input buffer via the VART API, and then the runner is called to perform the inference task. At this time, the DPU accesses the memory buffer data and performs operations on all layers included in the graph. After the DPU completes the operation, it places the result back into the memory buffer, removes the data from the memory buffer, and places it back into the user-state memory space.

[0090] For each input image, the above steps of preprocessing, data transfer, inference, and result acquisition are repeated. Multithreading and asynchronous execution are used to optimize data transfer efficiency between the PS and PL. The Zynq PS has two CPU cores: the first is responsible for communication with the DPU and memory management, while the second is responsible for data preprocessing. A pipeline approach is used to further improve inference speed.

[0091] The neural network used is a classification network, used to identify whether an image is of a typhoon. First, the output is parsed. Based on the model's output probability, a threshold (0.5) is set to map the continuous output to a binary classification result of 0 or 1. The classification results are saved to the local file system as text files, labeled with the original data path, file name, and processing time. The results are then transmitted to a host computer via the network for subsequent analysis and storage.

[0092] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for identifying extreme weather conditions at the edge using microwave radiometer data, characterized by: The following steps are involved: 1) Obtain microwave radiometer remote sensing data; 2) Filtering the microwave radiometer remote sensing data obtained in step 1) and creating a basic data set based on the time and longitude and latitude of recorded extreme weather events; 3) Build a convolutional neural network based on the number and types of extreme weather conditions to be identified; Train convolutional neural networks and derive extreme weather recognition models; 4) Screening the data set used for the quantitative model and quantifying the exported model; 5) Use extreme weather recognition models to identify extreme weather; In step 1), remote sensing data is acquired from a remote sensing satellite equipped with a microwave radiometer. The remote sensing data includes brightness temperature information, timestamp, and latitude and longitude. The brightness temperature information includes the brightness temperature of the five channels of the microwave radiometer: 18.7 GHz-H channel brightness temperature, 18.7 GHz-V channel brightness temperature, 23.8 GHz-V channel brightness temperature, 37 GHz-H channel brightness temperature, and 37 GHz-V channel brightness temperature. In step 3), when building a convolutional neural network based on the number and type of extreme weather conditions to be identified, a deep learning model is constructed based on the residual network; The residual network is built based on basic-1block and basic-2block; The input matrix is ​​scaled to 224*224 pixels in the first two dimensions, and then passes through a 7*7 conv kernel, a Maxpool layer, a basic-1 block, three basic-2 blocks, and finally an FC layer; In step 3), when training the convolutional neural network, the basic dataset and the augmented dataset are used for training; the training process includes forward propagation, backpropagation and optimization, using the Cross-Entropy loss function and the Adam optimizer; The training process is organized into multiple epochs, each of which contains multiple iteration steps; In step 4), when quantizing the exported model, if a piece of data is input into the neural network constructed in step 3), and the output value of a neuron in the last FC layer of the network is greater than 0.85, then the data is considered representative data. All representative data in the training set are selected as candidate data sets for quantization. Assume there are k labels, among which N labels are selected from the i-th label. i representative data, then N min =min(N1,N2,…,N i ,…,N k ), randomly select N from the representative data of each label min data as a quantized dataset; the original model is converted from float32 to int8 format using Vitis-AI's PTQ quantization tool. During the quantization process, the vectorization tool inputs the quantized dataset, and the tool automatically counts the dynamic range distribution of activation values ​​and weights of each layer. Based on the statistical results, the scale factor and zero point of each layer are calculated based on the asymmetric quantization scheme. Then, the original FP32 model weights and activation values ​​are quantized according to the following formula: Q = round(FP32 / S) + Z, where S is the scale factor and Z is the zero point.

2. The edge extreme weather identification method applicable to microwave radiometer data according to claim 1 is characterized by: In step 2), according to the recorded time and longitude and latitude of the extreme weather, in the process of screening the microwave radiometer remote sensing data obtained in step 1), positive sample data and negative sample data are screened; The specific process of screening positive sample data includes: 201) Filtering remote sensing data packets containing brightness temperature information according to the time and longitude and latitude of extreme weather occurrence, and unpacking the data packets; 202) Filtering scanning points located in extreme weather areas based on latitude and longitude information, assigning these scanning points to multiple layers according to microwave bands and polarization modes, and generating microwave images containing multiple band and polarization information; 203) Convert the brightness temperature data of the scan points to Kelvin and store them in the HDF5 file in float16 format; The process of screening negative sample data includes: 204) selecting a time and area where the extreme weather to be identified does not exist, and repeating steps 201) to 203) to generate negative sample data; The process of making a basic dataset includes: 205) The microwave images with and without extreme weather are stored in folders with different names, and the folder names are used as data labels.

3. The edge extreme weather identification method applicable to microwave radiometer data according to claim 1 is characterized in that: After step 2) is completed, the basic data set is enhanced, including scaling, rotating, stretching, cropping or padding the microwave image in the first and second dimensions. The expanded data set is stored in the storage format of the basic data set.

4. The edge extreme weather identification method applicable to microwave radiometer data according to claim 1 is characterized in that: In step 4), the dataset contains an equal proportion of extreme weather images and common non-extreme weather images, and the total amount of data is 10% of the total training set. The floating-point model is quantized to an int8 model. During the quantization process, the quantized dataset is imported, the activation values ​​and weight information are counted, the scale factor and zero point are determined, and then verification and fine-tuning are performed.

5. The edge extreme weather identification method applicable to microwave radiometer data according to claim 2, characterized in that: In step 201), let the time when extreme weather occurs be t, and the longitude and latitude of the center of extreme weather at that time be [λ, φ]. min ,t max ] time range, the range of extreme weather is {[λ min ,λ max ]、[φ min ,φ max ]}, where t min =t-15min, t max =t+15min;λ min =λ-7.5°,λ max =λ+7.5°;φ min =φ-7.5°,φ max =φ+7.5°; λ is the longitude of the extreme weather center, φ is the latitude of the extreme weather center, t min is the earliest time when extreme weather occurs, t max For the moment when extreme weather disappears, min is the westernmost longitude of the extreme weather area, λ max is the easternmost longitude of the extreme weather area, φ min is the southernmost latitude of the extreme weather area, φ max The northernmost latitude of the region where extreme weather occurs; For the downloaded remote sensing data package, first analyze the file name and select the acquisition time t i Satisfy t min <t i <t max Data packets, record the names of these data packets in an array, and open the data packets in sequence; In step 202), based on the time and longitude and latitude of the extreme weather, remote sensing data packets containing extreme weather are screened out, the data packets are unpacked, and brightness temperature information is extracted; The brightness temperature information includes the five channels of the microwave radiometer. The brightness temperature data of these channels are resampled to the same set of grid points to ensure that there are brightness temperature data of five channels at each longitude and latitude. According to the longitude and latitude of the extreme weather eye, all scanning points within a square longitude and latitude grid with a span of 15° centered on that longitude and latitude are selected; In step 203), the brightness temperature data of the selected scanning points are converted from count values ​​to Kelvin; the extreme weather area, i.e. {longitude∈[λ min ,λ max ], latitude∈[φ min ,φ max ]}, all scanning points are arranged according to the longitude and latitude information to obtain a set of three-dimensional arrays; the obtained three-dimensional arrays are stored in the HDF5 file in float16 format; Perform the above operations on multiple extreme weather time and longitude and latitude regions to obtain multiple HDF5 files, which constitute the positive sample data; In step 204), the microwave radiometer raw data is filtered according to the time and area that do not contain extreme weather, and steps 201) to 204) are repeated to generate negative sample data.

6. The edge extreme weather identification method applicable to microwave radiometer data according to claim 3, characterized in that: After step 2) is completed, when performing data augmentation on the basic data set, the images in the basic data are randomly scaled within the range of [-0.2, 0.2]; the images are randomly rotated within the range of [-90°, 90°]; the images are randomly stretched in the X or Y direction; and finally, the images are padded with zeros to make the final image rectangular. The enhanced data storage format is consistent with the basic dataset.

7. An edge extreme weather recognition system for microwave radiometer data, characterized by: When building the system, a deep learning processor suitable for the edge extreme weather recognition method according to any one of claims 1 to 6 is selected, and the model is compiled using a compiler suitable for the processor; The compiled model is imported into the deep learning dedicated processor on the edge; the model is run to perform extreme weather identification on the edge device.

8. The edge extreme weather identification system applicable to microwave radiometer data according to claim 7, characterized in that: When selecting a deep learning processor, configure a DPU adapted for the Zynq chip; add DPUCZDX8G to the Vivado project and select the B4096 architecture; adjust the DPU's AXI interface bandwidth based on the Zynq PS architecture; and enable support for the leaky-relu layer in the DPU configuration file. After the synthesis step, in the floorplanning tool, select the location on the FPGA chip closest to the PS end for the DPU; In the implementation strategy options, select the Explore strategy; add the -SSI_HighUtilSLRs directive; After completing the IP configuration, generate the bitstream file .bit containing the DPU IP core; When using the petalinux configuration kernel to create a Linux system, configure the kernel, add driver support for the DPU, and add the DPU path in the device tree; After the compilation is complete, the system image BOOT.BIN and image.ub containing the DPU driver are generated, and the system image is burned to the EMMC memory of the ZYNQ chip; Call the Vitis-AI Runtime API to load the model, perform scaling, normalization, and DPU format conversion on the input image, allocate kernel-state continuous memory buffers, and drive the DPU to perform graph operations through the VART API, implementing the input→inference→output data flow; The dual-core CPU performs asynchronous pipeline control to build a three-level pipeline of preprocessing, transmission, and inference; CPU core 1 manages DPU communication and memory buffering, and CPU core 2 performs data preprocessing.

Images